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EXECUTIVE  SUMMARY 


Instructional  Science  and  Development,  Inc.  'IISD,  Inc.r^rias  been  under 
contract  to  Headquarters,  Military  Airlift  Command  (HQ  MAC)  to  develop. 
Implement,  and  validate  a  prototype  Air  Transportation  ComputerySased  Training 
(ATCBT)  system  for  use  In  providing  Initial,  recurring,  and  upgrade  training 
for  MAC  aerial  port  (AFSC  605XX)  personnel.  The  goals  established  for  the 
prototype  training  system  are  to;  (1)  support  personnel  development,  (2) 
qualify  personnel  to  perform  work  assignments,  (3)  establish  minimum 

requirements  for  personnel  qualification  for  specific  duties,  and  (4)  Identify 
personnel  job  training  needs. 

To  achieve  these  training  goals,  the  prototype  CBT  system  was  designed  to 
meet  the  following  criteria: 

1.  Integrate  training  content  from  the  formal  schools,  on-the-job 
training  (OJT),  Career  Development  Courses  (CDCs),  and  the  Unit 
Learning  Centers  (ULCs). 

2.  Present  Interactive,  performance-based  training  at  the  9th  grade 

reading  level. 

3.  Develop  competency-based  training  using  standard  Instructional 
Systems  Development  (ISD)  procedures. 

4.  Provide  self-paced  training  modules  to  minimize  supervisory 

Involvement. 

5.  Provide  job  specific  training  Information  for  trainees  and  training 

management  Information  for  supervisors. 

6.  Deliver  training  to  the  work  station  where  It  Is  easily  accessible. 

7.  Standarize  training  across  aerial  ports  to  meet  training  requirements 

for  both  active  and  reserve  forces.  '  ^ 

The  contract  Statement  of  Work  specified  three  ATCBT  system  components: 
delivery,  production,  and  management.  The  operational  concept  for  the  training 
environment  Is  decentralized  CBT  delivery  components  at  the  aerial  port  work 
stations,  a  centralized  CBT  production  component  at  the  MAC  Module  Development 
Center  (MOC),  and  automatic  generation  of  training  management  Information  for 
the  local  aerial  ports,  the  MDC,  and  other  managerial  units  such  as  HQ  MAC  and 
Numbered  Air  Forces.  The  purpose  of  the  prototype  Implementation  was  to 
evaluate  this  operational  concept  at  three  aerial  ports,  each  with  differing 
mission  requirements,  and  at  the  MOC. 

Five  tasks  were  scheduled  for  completion  during  the  first  15  months  of  the 
three-year  contract.  Task  1  was  to  Identify  the  prototype  system  design  and 
develop  plans  for  Implementation.  Task  2  was  to  perform  a  task  and  media 
analysis  to  define  the  training  requirements.  Tasks  1  and  2  were  performed 
concurrently  to  ensure  that  the  system  was  designed  to  meet  the  training 
requirements.  Task  3  was  to  Implement  the  production  component  at  the  MDC 
located  at  Travis  AFB,  California.  Task  4  was  to  Implement  the  prototype 
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delivery  system  at  three  aerial  ports  sites:  Charleston  AFB,  South  Carolina; 
Dyess  AFB,  Texas;  and  Rhein  Main  AB,  Germany.  Task  5  was  to  validate  and 
evaluate  the  ATCBT  system  at  the  three  field  sites  and  the  HOC. 

The  purpose  of  the  final  report  is  to  document  the  results  of  the 
prototype  evaluation,  present  recommendations  on  the  basis  of  the  evaluation 
results,  and  provide  a  command-wide  implementation  plan.  This  report  is 
organized  into  three  major  sections.  Section  1.0  describes  the  project 
background  and  provides  a  description  of  the  activities  conducted  during  each 
of  the  five  tasks.  Section  2.0  presents  the  prototype  evaluation  results  and 
recommendations.  It  is  organized  by  the  three  ATCBT  system  components: 
delivery,  management,  and  production.  Section  3.0  provides  the  command-wide 
implementation  plan. 

Background 

At  the  request  of  the  Secretary  of  the  Air  Force,  a  series  of  studies  was 
performed  by  the  USAF  Scientific  Advisory  Board,  the  OCS/Air  Transportation, 
and  a  commercial  firm  under  contract  to  MAC.  These  studies  identified 
deficiencies  in  training  the  Air  Transportation  career  fields  and  provided 
recommendations  to  improve  the  training.  The  training  deficiencies  in  the 
aerial  port  squadrons  fell  into  several  major  categories:  (1)  the  need  to 
integrate  the  Unit  Learning  Center  (ULC)  and  on-the-job  training  (OJT) 
functions,  (2)  the  need  to  provide  standardized,  performance-based  training  and 
meaningful  training  data  to  management  at  various  levels,  and  (3)  the  need  to 
upgrade  the  training  technology  used  in  the  field.  Slide/tape  training  is 
currently  used  at  the  ULC.  This  training  technology  has  the  following 
drawbacks:  (1)  it  is  a  passive  rather  than  an  active  learning  environment,  (2) 
it  does  not  provide  any  feedback  to  the  trainee  or  the  supervisor,  (3)  it  is 
difficult  to  keep  current,  and  (4)  it  is  not  easily  accessible  from  the  work 
stations.  The  prototype  ATCBT  evaluation  project  was  structured  to  correct 
these  training  deficiencies. 

Prototype  System  Design 

The  operational  concept  of  decentralized  CBT  delivery  components,  a 
centralized  production  component,  and  a  management  component  which  functions  in 
the  background  was  the  basis  for  the  prototype  system  design.  To  satisfy  this 
concept,  it  was  necessary  to  identify  a  basic  system  configuration  with  enough 
flexibility  to  be  tailored  to  meet  the  specific  ATCBT  training  environment.  A 
survey  of  packaged  CBT  systems  (those  commercially  available  systems  providing 
both  hardware  and  software)  clearly  convinced  the  design  team  that  a  packaged 
CBT  system  would  force  the  training  environment  to  fit  the  hardware  and 
software  rather  than  the  other,  more  appropriate,  direction  of  fitting  the 
system  to  the  training  needs.  A  "systems  integration"  approach  was  adopted  in 
the  belief  that  integrating  commercially  available,  off-the-shelf  components 
would  best  allow  a  design  which  served  the  intended  user. 

Within  this  systems  integration  approach,  ISD,  Inc.  established  the 
following  design  criteria: 


1.  Transportability.  The  capability  to  execute  the  software  on  a 
"family"  of  compatible  microcomputers  was  required  to  eliminate 
dependency  on  a  single  manufacturer.  The  flexibility  to  modify  the 
basic  configuration  to  meet  the  needs  of  different  aerial  port  sizes 
and  missions  was  seen  as  mandatory. 

2.  User-Friendly.  New  skills  for  using  the  training  were  undesirable. 
Student  interaction  should  require  minimal  skills  and  the  training 
should  be  accessible  at  the  work  station.  The  use  of  non-programmers 
as  authors  was  a  necessity  in  order  to  allow  development  and 
maintenance  of  lessons  in-house  by  MAC  air  transportation  personnel. 

3.  Adequate  Training.  A  match  between  system  features  and  critical 
instructional  characteristics  (including  media)  was  required  to 
provide  effective,  low-cost  training  for  the  specific  training 
applications.  Training  should  tie  directly  to  Job  requirements  and 
be  standard  across  the  career  field. 

4.  Design- to-Cost.  The  approach  was  to  minimize  development  costs 
through  use  of  off-the-shelf  hardware/software  and  minimize  training 
equipment  maintenance  and  operation  costs. 

Task  and  Media  Analysis 

A  task  analysis  was  performed  to  identify  training  requirements  and  to 
select  tasks  for  training  for  both  the  prototype  .evaluation  and  command-wide 
implementation.  A  media  analysis  was  performed  to  identify  the  media 
attributes  required  to  meet  the  instructional  characteristics  and  to  define  the 
mix  of  media  to  accomplish  the  training  requirements. 

In  order  for  the  ATCBT  system  to  be  accepted  and  useful  at  the  aerial 
ports,  it  had  to  specifically  meet  AFSC  605XX  training  requirements.  These 
training  requirements  are  published  in  the  Specialty  Training  Standards  (STSs) 
that  designate  tasks  which  must  be  trained  in  order  to  progress  in  the  career 
field.  As  outlined  in  the  STSs  for  AFSC  605XX,  the  primary  focus  of  OJT  is  to 
take  the  basic  Air  Transportation  Training  Command  (ATTC)  graduate  entering  the 
squadron  at  a  3-skill  level  to  a  S-skill  level.  The  introduction  of  delivery 
stations  at  the  work  centers  provides  OJT  supervisors  with  a  resource  to 
accomplish  this  training.  The  CBT  lessons  used  in  combination  with  hands-on 
training  minimize  the  time  required  by  the  OJT  supervisor  to  ensure  proficiency 
requirements  are  satisfied.  Although  the  primary  intent  of  ATCBT  instruction 
is  to  provide  3-  to  5-skill  level  training,  it  can  also  be  used  to  provide 
proficiency  training,  recurring  training,  refresher  training  for  other  levels, 
and  cross-training  at  the  7-skill  level  (Air  Transportation  Supervisor— AFSC 
60572). 

Production  Component  Implementation 

Implementation  of  the  production  component  consisted  of  a  number  of 
subtasks.  These  included: 

1.  Development  of  the  authoring  environment  for  the  MOC  to  use. 

2.  Development  of  the  prototype  ATCBT  lessons. 


3.  Validation  of  the  prototype  lessons. 

4.  Installation  of  three  ATCBT  systems  at  Travis  AFB. 

5.  Training  the  MDC  staff  In  analysis,  design,  and  authoring  procedures 
and  In  the  use  of  the  data  management  functions. 

The  first  three  subtasks  are  described  In  detail  In  Section  1.0.  Training 
the  MDC  staff  was  an  ongoing  activity.  ISO,  Inc.  personnel  worked  closely  with 
the  MDC  personnel  at  all  stages  of  the  production  component  Implementation. 
The  MDC  Inputs  were  essential  In  the  development  of  a  useful  authoring 
environment.  Their  expertise  and  review  comments  were  also  necessary  In  the 
development  and  validation  of  the  prototype  lessons. 

Prototype  Site  Implementation 

Three  aerial  ports  with  differing  missions  were  specified  as  prototype 
Implementation  sites  to  test  the  ATCBT  concept  of  operation.  Charleston  AFB, 
South  Carolina  represents  a  major  strategic  port  responsible  for  moving  a  large 
number  of  passengers  and  cargo  In  support  of  the  MAC  mission.  Dyess  AFB,  Texas 
represents  a  mobile  aerial  port  that  provides  a  training  environment  unique  to 
mobility  planning  and  operations.  Rhein  Main  AB,  Germany  represents  a  major 
strategic  port  operating  In  a  non-CONUS  environment. 

An  ATCBT  Implementation  Plan  was  developed  to  define:  (1)  ATCBT  training 
organization,  roles  and  responsibilities,  (2)  training  and  Job  aids  for  ATCBT 
users,  (3)  facility  preparation  requirements.  Including  environmental 
considerations,  power  requirements,  base  communications  requirements,  (4) 
maintenance  support  requirements.  Including  preventive  and  corrective 
maintenance  procedures,  and  (5)  the  plan  for  prototype  system  validation.  In 
addition,  ISO,  Inc.  prepared  a  User's  Guide  for  Prototype  Evaluation  and 
Validation  for  use  by  the  ULC  monitor. 

The  ATCBT  systems  were  Installed  at  the  prototype  sites  during  the  period 
June  through  August  1985.  This  schedule  allowed  up  to  26  weeks  for  the 
evaluation  period.  ISD,  Inc.  personnel.  In  conjunction  with  personnel  from  HQ 
MAC  and  the  MDC,  spent  a  minimum  of  one  week  per  site  for  ATCBT  system 
Implementation.  Activities  Included: 

1.  Physical  Installation  of  the  hardware  and  software. 

2.  System  testing. 

3.  Training  the  ULC  monitor  and  designated  personnel  on  procedures  for 
taking  lessons,  generating  reports,  managing  training  data, 
preventive  and  corrective  maintenance,  and  collecting  validation 
data. 

4.  Briefing  aerial  port  squadron  supervisors  on  the  availability  of  the 
training  system  at  their  work  stations. 


Prototype  System  Evaluation 

The  last  task  was  to  validate  and  evaluate  the  prototype  system  at  the 
three  aerial  ports  and  the  MOC.  The  purpose  of  the  evaluation  was  to  determine 
the  effectiveness  of  the  prototype  ATCBT  system  In  providing  Initial, 
recurring,  and  upgrade  training  for  MAC  aerial  port  personnel.  The  evaluation 
was  not  designed  to  measure  the  effectiveness  of  CBT  as  a  medium  versus  other 
media.  It  was  designed  to  evaluate  the  effectiveness  of  Integrating  CBT  Into 
the  OJT  environment. 

Evaluation  of  the  prototype  delivery  and  management  components  required 
Inputs  on  use  and  acceptance  by  trainees,  the  ULC  monitor,  and  the  work  station 
supervisors.  Four  data  collection  tools  were  provided:  (1)  automatic  data 
collection  provided  as  part  of  the  prototype  CBT  lessons,  (2)  questionnaires 
administered  to  trainees  after  completion  of  Individual  lessons,  (3)  surveys 
administered  to  ULC  supervisors,  and  (4)  system  use  logs.  Evaluation  of  the 
production  component  Included  Interviews  with  HOC  personnel  and  an  ATCBT 
authoring  system  log  maintained  throughout  the  evaluation  period. 

The  results  Indicated  that: 

1.  CBT  Is  effective  In  delivering  OJT  Instruction  at  local  work 

stations. 

2.  Reliability  and  maintainability  of  the  CBT  system  components  In  a  24- 
hour  operation  at  work  stations  Is  satisfactory. 

3.  The  centralized  CBT  lesson  production  component  Is  operational. 

4.  The  ATCBT  authoring  environment  can  be  used  by  non-prograntners. 

5.  The  management  component  for  collecting  data  In  the  background  works 
In  several  situations,  but  Is  limited  In  bases  which  have  mechanical 
PBXs. 

6.  Collecting,  consolidating,  and  disseminating  training  management 

Information  In  the  form  of  printed  reports  at  the  local  level  Is 
useful.  However,  the  OJT  and  section  supervisors  require  more 

detailed  training  Information. 

7.  The  prototype  ATCBT  system  minimizes  supervisor  time  needed  for 
training,  requires  no  additional  manpower  and  little  additional 
skills  training. 

Command-Wide  Implementation  Plan 

The  proposed  plan  for  command-wide  Implementation  of  the  ATCBT  system  Is 
Is  based  on  the  requirements  In  the  contract  Statement  of  Work,  the  results  and 
recommendations  of  the  prototype  ATCBT  system  evaluation,  and  discussions  with 
HQ  MAC  personnel.  It  represents  the  best  current  efforts  of  ISO,  Inc.  to 
specify  all  of  the  events  which  must  occur  and  propose  a  tentative  schedule  to 
accomplish  these  events.  Implementation  of  the  plan  Is  subject  to  approval  by 
HQ  MAC. 


The  scope  of  the  conmand-wl de  Implementation  is  to  Install  ATCBT  systems 
at  41  sites  worldwide  with  an  inventory  of  approximately  200  hours  of  CBT 
Instruction  to  provide  initial,  recurring,  and  upgrade  training  for  AFSC  605XX 
personnel.  The  proposed  schedule  for  ATCBT  installation  command-wide  covers 
the  period  from  1  April  1986  to  15  February  1988. 
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SECTION  1.0 

PROTOTYPE  ANALYSIS  AND  DEVELOPMENT 

Instructional  Science  and  Development,  Inc.  (ISD,  Inc.)  has  been  under 
contract  to  Headquarters,  Military  Airlift  Connand  (HQ  MAC)  to  develop. 
Inclement,  and  validate  a  prototype  Air  Transportation  Computer-Based  Training 
(ATCBT)  system  for  use  In  providing  Initial,  recurring,  and  upgrade  training 
for  MAC  aerial  port  (AFSC  605XX)  personnel.  The  goals  established  for  the 
prototype  training  system  are  to:  (1)  support  personnel  development,  (2) 

qualify  personnel  to  perform  work  assignments,  (3)  establish  minimum 

requirements  for  personnel  qualification  for  specific  duties,  and  (4)  Identify 
personnel  Job  training  needs. 

To  achieve  these  training  goals,  the  prototype  CBT  system  was  designed  to 
meet  the  following  criteria: 

1.  Integrate  training  content  from  the  formal  schools,  on- the- Job 
training  (OJT),  Career  Development  Courses  (CDCs),  and  the  Unit 
Learning  Centers  (ULCs). 

2.  Present  Interactive,  performance- based  training  at  the  9th  grade 

reading  level. 

3.  Develop  competency-based  training  using  standard  Instructional 

Systems  Development  (ISO)  procedures. 

4.  Provide  self-paced  training  modules  to  minimize  supervisory 

Involvement. 

5.  Provide  Job  specific  training  Information  for  trainees  and  training 

management  Information  for  supervisors. 

6.  Deliver  training  to  the  work  station  where  It  Is  easily  accessible. 

7.  Standarize  training  across  aerial  ports  to  meet  training  requirements 
for  both  active  and  reserve  forces. 

The  contract  Statement  of  Work  specified  three  ATCBT  system  components: 
delivery,  production,  and  management.  The  operational  concept  for  the  training 
environment  Is  decentralized  CBT  delivery  components  at  the  aerial  port  work 
stations,  a  centralized  CBT  production  component  at  the  MAC  Module  Development 
Center  (MDC),  and  automatic  generation  of  training  management  Information  for 
the  local  aerial  ports,  the  MDC,  and  other  managerial  units  such  as  HQ  MAC  and 
Numbered  Air  Forces.  The  purpose  of  the  prototype  Implementation  was  to 
evaluate  this  operational  concept  at  three  aerial  ports,  each  with  differing 
mission  requirements,  and  at  the  MDC. 

Five  tasks  were  scheduled  for  completion  during  the  first  15  months  of  the 
three-year  contract.  Task  1  was  to  Identify  the  prototype  system  design  and 
develop  plans  for  Implementation.  Task  2  was  to  perform  a  task  and  media 
analysis  to  define  the  training  requirements.  Tasks  1  and  2  were  performed 
concurrently  to  ensure  that  the  system  was  designed  to  meet  the  tralnlno 
requirements.  Task  3  was  to  Implement  the  production  component  at  the  MDC 
located  at  Travis  AFB,  California.  Task  4  was  to  Implement  the  prototype 
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delivery  system  at  three  aerial  ports  sites:  Charleston  AFB,  South  Carolina; 
Dyess  AFB,  Texas;  and  Rhein  Main  AB.  Germany.  Task  5  was  to  validate  and 
evaluate  the  ATCBT  system  at  the  three  field  sites  and  the  MDC. 

The  purpose  of  this  report  Is  to  document  the  results  of  the  prototype 
evaluation,  present  recommendations  on  the  basis  of  the  evaluation  results,  and 
provide  a  command-wide  Implementation  plan  for  the  follow-on  contract  period. 
This  report  Is  organized  into  three  major  sections.  The  remainder  of  Section 
1.0  describes  the  project  background  and  provides  a  description  of  the 
activities  conducted  during  each  of  the  five  tasks.  Section  2.0  presents  the 
prototype  evaluation  results  and  recoomendatlons.  It  Is  organized  by  the  three 
ATCBT  system  components:  delivery,  management,  and  production.  Section  3.0 
provides  the  command-wide  Implementation  plan. 

1.1  Background 

At  the  request  of  the  Secretary  of  the  Air  Force,  a  series  of  studies  was 
performed  by  the  USAF  Scientific  Advisory  Board,  the  DCS/AIr  Transportation, 
and  a  commercial  firm  under  contract  to  MAC.  These  studies  Identified 
deficiencies  In  training  the  Air  Transportation  career  fields  and  provided 
recommendations  to  Improve  the  training.  The  training  deficiencies  in  the 
aerial  port  squadrons  fell  Into  several  ma^or  categories:  (1)  the  need  to 
Integrate  the  Unit  Learning  Center  (ULC)  and  on-the-job  training  (OJT) 
functions,  (2)  the  need  to  provide  standardized,  performance-based  training  and 
meaningful  training  data  to  management  at  various  levels,  and  (3)  the  need  to 
upgrade  the  training  technology  used  In  the  field.  Slide/tape  training  Is 
currently  used  at  the  ULC.  This  training  technology  has  the  followlnq 
drawbacks:  (1)  It  Is  a  passive  rather  than  an  active  learning  environment,  (2) 
It  does  not  provide  any  feedback  to  the  trainee  or  the  supervisor,  (3)  It  Is 
difficult  to  keep  current,  and  (4)  It  Is  not  easily  accessible  from  the  work 
stations.  The  prototype  ATCBT  evaluation  project  was  structured  to  correct 
these  training  deficiencies. 

1.2  Prototype  System  Design 

The  operational  concept  of  decentralized  CBT  delivery  components,  a 
centralized  production  component,  and  a  management  component  which  functions  In 
the  background  was  the  basis  for  the  prototype  system  design.  To  satisfy  this 
concept.  It  was  necessary  to  Identify  a  basic  system  configuration  with  enough 
flexibility  to  be  tailored  to  meet  the  specific  ATCBT  training  environment.  A 
survey  of  packaged  CBT  systems,  those  conmerclally  available  systems  providing 
both  hardware  and  software,  clearly  convinced  the  design  team  that  a  packaged 
CBT  system  would  force  the  training  environment  to  fit  the  hardware  and 
software  rather  than  the  other,  more  appropriate,  direction  of  fitting  the 
system  to  the  training  needs.  A  "systems  Integration"  approach  was  adopted  in 
the  belief  that  Integrating  commercially  available,  off-the-shelf  components 
would  allow  a  design  which  best  served  the  Intended  user. 

Within  this  systems  Integration  approach,  ISO,  Inc.  established  the 
following  design  criteria: 
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1.  Transportability.  The  capability  to  execute  the  software  on  a 
"family"  of  compatible  microcomputers  was  required  to  eliminate 
dependency  on  a  single  manufacturer.  The  flexibility  to  modify  the 
basic  configuration  to  meet  the  needs  of  different  aerial  port  sizes 
and  missions  was  seen  as  mandatory. 

2.  User-Friendly.  New  skills  for  using  the  training  were  undesirable. 
Student  interaction  should  require  minimal  skills  and  the  training 
should  be  accessible  at  the  work  station.  The  use  of  non-programmers 
as  authors  was  a  necessity  in  order  to  allow  development  and 
maintenance  of  lessons  in-house  by  MAC  air  transportation  personnel. 

3.  Adequate  Training.  A  match  between  system  features  and  critical 
instructional  (including  media)  characteristics  was  required  to 
provide  effective,  low-cost  training  for  the  specific  training 
applications.  Training  should  tie  directly  to  job  requirements  and 
be  standard  across  the  career  field. 

4.  Design-to-Cost.  The  approach  was  to  minimize  development  costs 
through  use  of  off-the-shelf  hardware/software  and  minimize  training 
equipment  maintenance  and  operation  costs. 

The  prototype  system  design  was  based  on  a  survey  of  commercially 
available  hardware  and  software,  including  tests  and  demonstrations,  and  the 
results  of  the  task  and  media  analysis  described  in  Section  1.3  below.  The 
media  analysis  indicated  that  the  use  of  medium  resolution  color  graphics  and 
keyboard  input  were  sufficient  to  meet  the  learning  objectives  defined  for 
ATCBT. 

To  meet  the  design  criteria  of  transportability  adequacy  and  design- to- 
cost,  an  IBM  PC  clone  was  selected  as  the  core  of  the  prototype  system.  This 
decision  was  based  on  the  following  factors. 

1.  Runs  under  MS-OOS  to  ensure  compatibility. 

2.  Faster  and  cheaper  than  IBM-PC  (based  upon  a  request  for  quotes  and 
GSA  schedules). 

3.  Runs  a  large  number  of  authoring  languages. 

Specifically,  the  ATAT  6300  personal  computer  was  selected  because  of  its 
highly  rated  color  display  monitors  and  maintenance  support. 

Likewise,  the  selection  of  the  authoring  software  was  based  on  the  design 
criteria  of  transportability,  user-friendliness,  and  design-to-cost. 
Specifically,  CDS  II  (Courseware  Design  System),  developed  by  Electronic 
Information  Systems,  Inc.,  was  selected  for  the  following  reasons: 

1.  Written  in  C  so  it  is  highly  transportable  across  a  number  of 
microcomputers. 

2.  Lessons  developed  on  one  system  run  on  other  systems. 

3.  Instructional  paradigms  are  not  fixed. 
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4.  Command  language  for  programmers  and  a  "me ta- language"  to  build 
authoring  Interactions  for  non-programmers. 

5.  Features  for  recording  specified  data  which  can  be  processed  by 
tailored  data  base  management  programs. 

6.  Moderate  costs  for  authoring  and  delivery  software. 

As  shown  In  Table  1-1,  the  delivery  component  Is  designed  to  represent  a 
standardized  configuration  which  can  be  scaled  to  any  size  port.  The 
production  component  parallels  the  delivery  component,  but  Includes  features 
specific  to  the  authoring  process.  The  management  component  Is  based  on  the 
delivery  component,  but  Includes  several  additional  features  to  permit  the 
transmission  of  data  to  points  outside  the  port  and  generation  of  reports  for 
distribution  to  the  port.  This  Includes  a  draft  quality  printer,  modem,  data 
management  software  (dBase  II  for  the  prototype),  communications  software,  and 
the  CONTROL  software  developed  by  ISO,  Inc. 

The  production  component  Includes  the  capability  to  develop  graphics  In 
support  of  the  lesson  content.  The  cost  and  complexity  of  the  graphics 
software  available  to  run  on  the  AT&T  6300  under  MS-DOS,  as  well  as  the  options 
for  Input  devices,  varied  considerably.  PC  Paint  (developed  by  Mouse  systems, 
Inc.)  was  selected  for  the  following  reasons: 

1.  Resolution  supported  the  training  requirements. 

2.  Compatible  with  AT&T  6300  standard  graphics  board. 

3.  Supports  optical  mouse  for  Input  without  additional  interface  boards 
or  power  requirements. 

4.  Creates  picture  files  accessible  by  CDS  II  authoring  software. 

5.  Picture  file  storage  compressed  to  save  space  on  disk. 

6.  Provides  Icon  menu  command  structure. 

7.  Low  cost. 

Figure  1-1  graphically  Illustrates  the  ATCBT  prototype  system  design.  The 
development  and  maintenance  of  CBT  lessons  by  non-progranner  MAC  personnel  Is 
accomplished  through  the  production  component  at  the  MDC.  However,  It  Is  the 
management  component  that  makes  the  ATCBT  system  unique.  A  significant  feature 
Is  the  transmission  of  data  from  the  delivery  stations  at  the  work  centers  to 
the  manaaement  component  at  the  ULC,  and  subsequently  through  the  Defense  Data 
Network  (DDN)  to  the  MDC  (and  HQ  MAC  as  requested).  The  system  was  designed  to 
use  Class  C  telephone  lines  and  standard  PBX  switching  available  at  the 
prototype  sites. 
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Table  1-1 


ATCBT  Prototype  System  Design 


Component 

Configuration 

Del i very 

AT4T 

6300  Personal  Computer 

0 

MS-DOS 

0 

Single  floppy  disk  drive 

0 

10  megabyte  hard  disk  drive 

0 

640K  RAM 

0 

Keyboard 

0 

320x200  resolution  color  graphics  monitor 

0 

CDS  II  delivery  software 

0 

Modem 

Management 

ATiT 

6300  Personal  Computer 

0 

MS-DOS 

0 

Single  floppy  disk  drive 

0 

10  megabyte  hard  disk  drive 

0 

640K  RAM 

0 

Keyboard 

0 

320x200  resolution  color  graphics  monitor 

0 

CDS  II  delivery  software 

0 

dBase  II  software 

0 

Communications  software 

0 

CONTROL  software 

0 

Draft  quality  printer 

0 

Modem 

Production 

ATiT 

6300  Personal  Computer 

0 

MS-DOS 

0 

Dual  floppy  disk  drives 

0 

640K  RAM 

0 

Keyboard 

0 

320x200  resolution  color  graphics  monitor 

0 

PC  Paint  software  and  Mouse  System  mouse  Input  device 

0 

COS  II  authoring  software 
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HQ  MAC 


I 

Figure  l-l. 


ATCBT  System  Design 


I 


1-6 


As  a  trainee  completes  a  lesson  at  the  work  station,  the  data  is  recorded 
and  automatically  sent  to  the  ULC.  The  recording  and  reporting  of  such 
information  is  not  visible  to  the  trainee.  The  data  is  consolidated  at  the  ULC 
as  a  master  file.  Then,  the  ULC  monitor,  using  a  menu  of  report  options, 
generates  reports  for  the  various  levels  of  management,  including  the  OJT 
supervisor.  The  ULC  monitor  also  has  the  data  management  capability  to  add 
files,  edit  files,  and  purge  files  as  required.  After  data  has  been 
consolidated  and  reported  at  the  local  level,  it  is  periodically  sent,  via 
electronic  mail  on  DON,  to  the  HOC.  The  HOC  then  uses  this  management  data  to 
determine  the  requirements  for  new  lessons  and  revisions  to  existing  lessons. 

As  shown  in  Figure  1-1,  there  is  also  a  downward  transmission  of 
information  from  the  ULC  to  the  work  stations.  This  is  in  the  form  of  lesson 
down-loading.  A  unique  feature  of  the  ATCBT  is  the  capability  to  search 
throughout  the  system  for  a  lesson  which  has  been  requested  by  a  trainee.  For 
example,  if  a  trainee  requests  a  specific  lesson  which  is  not  stored  at  that 
station,  the  system  will  find  where  the  lesson  is  stored  (perhaps  at  the  ULC 
master  file  of  lessons)  and  offer  the  trainee  the  option  of  down-loading  that 
lesson  for  use.  In  addition,  this  same  capability  allows  the  ULC  monitor  to 
send  lessons  to  every  delivery  station  to  update  lessons  or  add  new  lessons  to 
the  files. 

1.3  Task  and  Media  Analysis 

A  task  analysis  was  performed  to  identify  training  requirements  and  to 
select  tasks  for  training  for  both  the  prototype  evaluation  and  command-wide 
implementation.  A  media  analysis  was  performed  to  Identify  the  media 
attributes  required  to  meet  the  instructional  characteristics  and  to  define  the 
mix  of  media  to  accomplish  the  training  requirements. 

In  order  for  the  ATCBT  system  to  be  accepted  and  useful  at  the  aerial 
ports,  it  must  specifically  meet  AFSC  605XX  training  requirements.  These 
training  requirements  are  published  in  the  Specialty  Training  Standards  (STSs) 
that  designate  tasks  which  must  be  trained  in  order  to  progress  in  the  career 
field.  As  outlined  in  the  STSs  for  AFSC  605XX,  the  primary  focus  of  OJT  is  to 
take  the  basic  Air  Transportation  Training  Command  (ATTC)  graduate  entering  the 
squadron  at  a  3-skill  level  to  a  5-skill  level.  The  introduction  of  delivery 
stations  at  the  work  centers  provides  OJT  supervisors  with  a  resource  to 
accomplish  this  training.  The  CBT  lessons  used  in  combination  with  hands-on 
training  minimize  the  time  required  by  the  OJT  supervisor  to  ensure  proficiency 
requirements  are  satisfied.  Although  the  primary  intent  of  ATCBT  instruction 
is  to  provide  3-  to  5-skill  level  training,  it  can  also  be  used  to  provide 
proficiency  training,  recurring  training,  refresher  training  for  other  levels, 
and  cross-training  at  the  7-skill  level  (Air  Transportation  Supervisor— AFSC 
60572). 

Review  of  STS  605X1  for  the  Air  Cargo  Specialist  and  STS  605X0  for  the  Air 
Passenger  Specialist  resulted  in  a  preliminary  estimate  of  142  hours  of 
instruction  that  could  be  Implemented  on  CBT.  This  estimate  was  revised  and 
refined  during  the  prototype  evaluation.  The  revised  estimate  is  included  in 
Section  3.4.3  of  this  report. 
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The  training  requirements  identified  in  the  STSs  Mere  analyzed  In  order  to 
specify  the  critical  media  attributes  and  the  mix  of  media  necessary  to 
accomplish  OJT.  The  results  of  the  media  analysis  indicated  that  medium 
resolution  color  graphics  and  keyboard  input  Mere  sufficient  to  meet  the 
training  requirements.  Computer  controlled  audio  and/or  video  and  student 
input  devices  such  as  a  mouse,  light  pen,  or  touch  panel  exceeded  the 
requirements  to  provide  adequate  training.  The  media  mix  to  satisfy  OJT 
proficiency  levels  Mas  defined  as  the  ATCBT  lessons  in  combination  Mith  adjunct 
materials  (in  the  form  of  reference  manuals,  technical  documentation, 
illustrations,  handouts)  and  actual  hands-on  performance. 

1.4  Production  Component  Implementation 

Implementation  of  the  production  component  consisted  of  a  rumber  of 
subtasks.  These  included: 

1.  Development  of  the  authoring  environment  for  the  MDC  to  use. 

2.  Development  of  the  prototype  ATCBT  lessons. 

3.  Validation  of  the  prototype  lessons. 

4.  Installation  of  three  ATCBT  systems  at  Travis  AFB. 

5.  Training  the  MDC  staff  in  analysis,  design,  and  authoring  procedures 
and  in  the  use  of  the  data  management  functions. 

The  first  three  subtasks  are  described  in  detail  in  the  folloMing 
subsections.  Training  the  MDC  staff  Mas  an  ongoing  activity.  ISD,  Inc. 
personnel  Morked  closely  Mith  the  MDC  personnel  at  all  stages  of  the  production 
component  implementation.  The  MDC  Inputs  Mere  essential  in  the  development  of 
a  useful  authoring  environment.  Their  expertise  and  revieM  comments  were  also 
necessary  in  the  development  and  validation  of  the  prototype  lessons. 

ISD,  Inc.  personnel  conducted  a  number  of  hands-on  training  sessions  on¬ 
site  and  at  the  MDC  after  the  ATCBT  systems  Mere  installed  in  June  1985.  These 
sessions  Included:  development  of  lesson  specifications,  procedures  for 
authoring,  procedures  for  editing,  development  of  graphics,  use  of  DDN, 
generation  of  management  reports,  procedures  for  dOMnloading  data.  A  User 
Guide  documenting  these. procedures  Mas  also  provided. 

1.4.1  Development  of  the  Authoring  Environment 

As  stated  earlier,  one  of  the  most  important  criteria  in  designing  the 
ATCBT  system  Mas  the  development  of  an  authoring  capability  for  non- 
programmers.  It  Mas  essential  that  no  computer  programming  skills  be  required 
for  MDC  personnel  to  author  a  lesson.  In  addition,  the  lessons  needed  to  be 
based  on  a  standardized,  structured,  sequence  of  instructional  events,  as 
opposed  to  a  "free- form"  set  of  text  screens.  The  solution  Mas  to  provide  an 
authorino  "environment"  in  Mhich  the  author  Mould  operate.  This  environment 
Mould  (1)  prompt  an  author  for  a  specific  entry  (such  as  a  learning  objective, 
test  item,  or  correct  ansMer)  in  a  specific  sequence,  (2)  take  the  author's 
response  to  the  prompt,  and  (3)  automatically  generate  computer  "code"  that  is 
executed  Mhen  the  student  runs  the  lesson. 
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The  ability  of  COS  II  to  construct  an  authoring  "meta"  language  was  used 
to  design  the  authoring  sequence  of  prompts  that  would  generate  a  lesson.  The 
author  only  Inputs  text  and  graphic  file  names  In  response  to  the  authoring 
prompts.  The  format,  sequencing,  and  branching  capabilities  of  a  lesson  are 
already  set  and  are  produced  without  any  additional  Input  by  the  author.  An 
author  only  needs  to  respond  to  prearranged  prompts  In  order  to  construct  a 
lesson. 

The  structure  of  the  lesson.  Including  the  prearranged  prompts.  Is  called 
the  “prototype,"  “lesson  prototype,"  or  “authoring  prototype."  The  prototype 
Is  based  on  a  strategy  or  “paradigm"  for  presenting  effective  Instruction  via 
computers.  The  Instructional  paradigm  for  producing  the  standardized  ATCBT 
lessons  defined  a  set  of  Instructional  units.  These  units  were  divided  Into 
two  levels:  the  lesson  level  based  on  the  terminal  learning  objective  and  the 
segment  level  based  on  the  enabling  learning  objectives.  Each  segment  Includes 
discrete  chunks  of  Instruction  (Introduction,  explanation,  practice,  and  test) 
related  to  one  lesson  enabling  objective.  A  lesson,  by  definition,  contains 
more  than  one  segment. 

The  authoring  prototype  was  developed  to  not  only  reflect  this  governing 
Instructional  paradigm,  but  also  to  Include  an  Instructional  design  that  was 
sensitive  to  the  delivery  environment.  For  example,  segments  can  be  accessed 
Individually  by  a  student,  and  a  student  can  escape  at  any  point  In  the  lesson 
In  the  event  more  pressing  events  occur  at  the  work  station.  The  student  can 
then  come  back  and  continue  with  the  same  segment.  The  capability  to  access 
the  segments  Individually  also  maximizes  the  usefulness  of  the  lesson  exam. 
Students  receive  a  prescription  of  what  segments  need  to  be  taken  based  on 
their  performance  on  the  lesson  exam.  The  required  segments  can  then  be  taken 
rather  than  the  entire  lesson.  This  feature  fits  the  design  concept  of 
adequate  training,  since  students  can  view  only  those  portions  of  the  Usson 
that  are  required. 

Several  basic  conventions,  based  on  Instructional  design  principles,  were 
established  for  the  units  In  the  ATCBT  authoring  prototype.  The  authoring 
conventions  Include: 

0  Learning  objectives  that  Include  conditions,  behavior,  and  standards. 

0  Inclusion  of  lesson  prerequisites,  required  materials,  and  related 
STS  Items  at  the  start  of  the  program. 

0  A  student  option  to  take  the  lesson  exam  as  a  pretest  or  go  to 
specific  learning  segments  at  the  start  of  a  lesson. 

0  A  lesson  exam  that  Incorporates  Items  based  on  all  lesson  segments. 

0  Lesson  exam  feedback  that  displays  which  Items  were  missed  and  tags 
Individual  test  Items  to  their  related  segments. 

0  Lesson  segments  based  on  Identified  enabling  learning  objectives. 

0  Practice  exercises  that  are  consistent  with  the  behavior  and 
conditions  specified  by  the  enabling  objectives. 
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0  Practice  Items  that  allow  the  student  to  make  three  attempts  for  the 
correct  response  before  advancing  to  the  next  Item. 

0  Segment  tests  that  are  matched  to  the  enabling  objective.  Thus,  the 
same  test  Item  format  as  that  for  the  practice  Items  Is  anticipated. 

0  Feedback  as  to  which  Items  are  missed  during  the  segment  test. 

0  Successful  completion  of  the  lesson  exam  Is  used  to  Indicate  mastery 

of  the  lesson  (terminal  learning  objective). 

The  Instructional  units  were  Initially  designed  by  ISO,  Inc.  and  reviewed 
by  the  MDC  staff  throughout  the  prototype  development  cycle.  The  MDC  staff  was 
Involved  In  the  design  of  authoring  Interfaces,  Including  the  wording  and 
appropriateness  of  explanations  and  prompts  within  the  prototype.  Their  Input 
contributed,  for  example,  to  the  Inclusion  of  established  exits  at  various 
points  In  the  authoring  process  and  to  the  segment  practice,  testing,  and 
feedback  structure. 

1.4.2  Development  of  Prototype  Lessons 

The  contract  Statement  of  Work  specified  that  eight  hours  of  CBT 
Instruction  be  developed  In  general  topic  areas  for  the  prototype  evaluation. 
The  topic  areas  for  these  lessons  were  further  defined  during  the  review  of  the 
STS  Items.  A  learning  analysis  was  performed  for  each  topic  to  determine  the 
entry  level  characteristics.  Identify  prerequisite  skills  and  knowledge,  and 
specify  the  lesson  content.  As  a  result,  nine  prototype  lessons  were 
specified— three  for  Air  Passenger  Specialists  and  six  for  Air  Cargo 
Specialists.  A  format  for  presenting  the  lesson  specifications  was  developed 
to  serve  as  guidelines  to  be  used  during  the  CBT  authoring  process. 

The  MDC  personnel  participated  In  task  selection  for  training  and  In  the 
construction  of  lesson  specifications.  The  purpose  of  their  Involvement  was 
not  only  to  utilize  their  expertise,  but  to  demonstrate  the  process  to  specify 
learning  objectives,  evaluation  strategies,  and  presentation  strategies  for 
potential  CBT  lessons.  The  goal  of  the  exercise  was  to  emphasize  the 
difference  between  CBT  lesson  development  and  the  more-familiar  sound-slide 
production.  In  which  scripts  and  storyboards  are  created  In  response  to  an 
assigned  topic,  with  little  consideration  for  testing/evaluation  strategies  and 
no  concern  for  display-response  Interactions. 

The  content  of  the  lessons  was  based  on  five  sources:  draft  technical 
manuals,  operational  orders.  Air  Force  regulations,  CDCs,  and  subject  matter 
specialists.  All  lessons  were  reviewed  at  various  stages  by  subject  matter 
specialists  from  HQ  MAC  and  the  MDC.  The  lessons  were  reviewed  for  appropriate 
use  of  terminology,  accuracy  of  content  and  practice/test  Items,  and 
completeness  of  Information.  Accompanying  graphics  were  developed  to 

Illustrate  points  of  Instruction  (e.g.,  hand  signals,  aircraft  safety  zones, 
equipment). 

1.4.3  Prototype  Lesson  Validation 

Three  stages  of  lesson  validation  occurred  during  the  prototype 
evaluation:  MDC  review,  small  group  validation  by  a  sample  of  the  target 
population,  and  field  Implementation.  The  MDC  staff  reviewed  the  lesson 
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specifications,  lesson  content,  and  test  Items  for  technical  correctness. 
Corrections  were  Incorporated  for  the  small  group  validation  by  personnel  from 
the  target  poulatlon  located  In  the  aerial  port  squadron  at  Travis  AFB. 

Validation  focused  on  the  lessons  “teachability"  as  opposed  to  lesson 
content.  A  lesson  could  be  technically  correct,  but  still  not  teach.  Ten  to 
12  students  were  assigned  to  take  three  lessons  each.  At  least  one  contractor- 
assigned  proctor  was  present  to  observe  and  answer  questions.  Any  questions, 
comments,  and  Inconsistencies  were  noted  as  students  progressed  through  each 
lesson  segnent.  These  notes  became  one  source  of  revision.  A  typical  example 
was  a  student  attempting  to  Input  an  apparently  correct  answer,  but  having  the 
answer  treated  by  the  software  as  Incorrect.  This  answer  was  not  considered  as 
a  possible  correct  answer  during  Initial  lesson  development  so  It  needed  to  be 
Included  as  a  potential  correct  answer  during  revision. 

Test  data  for  each  lesson  was  also  collected  In  the  background  as  students 
took  the  lessons.  The  data  Included  the  number  of  correct  and  Incorrect 
responses  per  test  Item  and.  If  the  response  was  Incorrect,  the  actual  response 
was  recorded.  This  data  pointed  to  questions  that  were  either  misleading  or 
not  well  covered  by  the  Information  presented  In  the  lesson.  Lesson  revisions 
were  made  on  the  basis  of  observed  problems/inconsistencles  and  the  test  Item 
analysis. 

Validation  continued  after  the  lessons  were  Implemented  at  the  prototype 
evaluation  sites.  Again  two  sources  of  Information  were  available  for  lesson 
revision:  comments  from  students  taking  the  lesson  and  test  Item  data. 
Revision  comments  from  the  field  were  received  by  the  HOC  staff  and  forwarded 
to  ISO,  Inc.  If  the  comment  required  Immediate  action  (e.g.,  a  possible 
correct  answer  not  treated  as  correct),  the  correction  was  made  and  updates 
were  sent  on  floppy  disks  to  the  prototype  evaluation  sites.  Comments  not 
requiring  Immediate  action  were  noted  as  a  lesson  maintenance  requirement  to  be 
performed  during  command-wide  Implementation.  Test  Item  data  was  also 
collected  and  analyzed  during  the  evaluation.  Again,  any  Indications  that  a 
potential  problem  existed  were  noted  as  a  lesson  maintenance  requirement  to  be 
completed  after  the  prototype  evaluation. 

1.5  Prototype  Site  Implementation 

Three  aerial  ports  with  differing  missions  were  specified  as  prototype 
Implementation  sites  to  test  the  ATCBT  concept  of  operation.  Charleston  AFB, 
South  Carolina  represents  a  major  strategic  port  responsible  for  moving  a  large 
number  of  passengers  and  cargo  In  support  of  the  MAC  mission.  Dyess  AFB,  Texas 
represents  a  mobile  aerial  port  that  provides  a  training  environment  unique  to 
mobility  planning  and  operations.  Rhein  Main  AB,  Germany  represents  a  major 
strategic  port  operating  In  a  non-CONUS  environment. 

Implementation  planning  prior  to  actual  Installation  at  each  aerial  port 
Included  a  number  of  activities.  As  part  of  the  planning  process,  a  survey  was 
sent  to  the  three  prototype  sites  and  to  the  command-wide  Implementation  sites 
to  gather  data  on  the  following: 


1.  Personnel /Work  Force  Description  (numbers  and  types). 

2.  Personnel  Scheduling  (work  station  shifts  and  OJT  schedule). 

3.  Work  Station  Layouts  (locations  and  distances  between  work  stations). 

4.  OJT  Training  (STS  tasks  and  other  tasks  requiring  training). 

This  data  was  used.  In  part,  to  Identify  the  number  of  systems  needed  at 
each  prototype  site.  ISO,  Inc.  developed  a  set  of  assumptions  based  on  the  mix 
of  aerial  port  personnel  working  In  cargo  and  In  passenger  services,  the  number 
of  personnel  by  shift,  the  number  of  student  contact  hours,  and  the  number  of 
hours  of  training  system  availability  to  determine  the  number  of  systems. 
Table  1-2  summarzies  the  configuration  Installed  at  each  prototype  site.  It 
should  be  noted  that  the  management  component  at  each  ULC  also  serves  as  a 
delivery  station  for  students  and  that  the  management  component  at  the  MOC 
provides  the  authoring  capability  of  a  production  component. 


Table  1-2 

Prototype  Implementation  Sites 


Prototype  Site 

Total 

Number 

Location 

Number  and  Type  of 
Component 

Travis  AFB 

3 

MDC 

1— Management 

HOC 

2— Production 

Charleston  AFB 

4  • 

ULC 

1— Management 

Passenger  Service 

1— Delivery 

Freight 

1— Delivery 

ATOC 

1— Delivery 

Dyess  AFB 

2 

ULC 

1— Management 

ULC 

1— Delivery 

Rhein  Main  AB 

4 

ULC 

1— Management 

Passenger  Service 

1— Delivery 

Freight 

1— Delivery 

ATOC 

1— Delivery 

An  ATCBT  Implementation  Plan  was  developed  to  define:  (1)  ATCBT  training 
organization,  roles  and  responsibilities,  (2)  training  and  Job  aids  for  ATCBT 
users,  (3)  facility  preparation  requirements.  Including  environmental 
considerations,  power  requirements,  base  communications  requirements,  (4) 
maintenance  support  requirements.  Including  preventive  and  corrective 
maintenance  procedures,  and  (5)  the  plan  for  prototype  system  validation.  In 
addition,  ISO,  Inc.  prepared  a  User's  Guide  for  Prototype  Evaluation  and 
Validation  for  use  by  the  ULC  monitor.  It  provided  detailed  procedures  for 
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operation  and  maintenance  of  the  ATCBT  system,  as  well  as  the  validation 
requirements  and  procedures. 

The  ATCBT  systems  were  installed  at  the  prototype  sites  between  June  and 
August  1985.  This  schedule  allowed  up  to  26  weeks  for  the  evaluation  period. 
ISO,  Inc.  personnel,  in  conjunction  with  personnel  from  HQ  MAC  and  the  MOC, 
spent  a  minimum  of  one  week  per  site  for  ATCBT  system  implementation. 
Activities  included: 

1.  Physical  installation  of  the  hardware  and  software. 

2.  System  testing. 

3.  Training  the  ULC  monitor  and  designated  personnel  on  procedures  for 

taking  lessons,  generating  reports,  training  data  management, 

preventive  and  corrective  maintenance,  and  validation  data 

collection. 

4.  Briefing  aerial  port  squadron  supervisors  on  the  availability  of  the 
training  system  at  their  work  stations. 

1.6  Prototype  System  Evaluation 

The  last  task  was  to  validate  and  evaluate  the  prototype  system  at  the 
three  aerial  ports  and  the  MDC.  The  purpose  of  the  evaluation  was  to  determine 
the  effectiveness  of  the  prototype  ATCBT  system  in  providing  initial, 
recurring,  and  upgrade  training  for  MAC  aerial  port  personnel.  The  evaluation 
was  not  designed  to  measure  the  effectiveness  of  CBT  as  a  medium  versus  other 

media.  It  was  designed  to  evaluate  the  effectiveness  of  integrating  CBT  into 

the  OJT  environment.  The  objectives  were  to  evaluate: 

1.  The  effectiveness  of  using  CBT  to  deliver  OJT  Instruction  at  local 
work  stations. 

2.  The  reliability  and  maintainability  of  the  CBT  system  components. 

3.  The  effectiveness  of  a  centralized  CBT  lesson  production  component. 

4.  The  ease  of  use  of  the  ATCBT  authoring  environment  by  non- 
programmers. 

5.  The  effectiveness  of  the  management  component  in  collecting  data  in 
the  background. 

6.  The  effectiveness  uf  collecting,  consolidating,  and  disseminating 
training  management  information— including  the  quality,  frequency, 
and  usefulness  of  the  reports. 

7.  The  capability  of  the  prototype  ATCBT  system  to  minimize  supervisor 
training  time  and  not  require  additional  manpower  or  skills  training. 

Evaluation  of  the  prototype  delivery  and  management  components  required 
inputs  on  use  and  acceptance  by  the  trainees,  the  ULC  monitors,  and  the  work 
station  supervisors.  Four  data  collection  tools  were  provided:  (1)  automatic 
data  collection  provided  as  part  of  the  prototype  CBT  lessons,  (2) 
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questionnaires  administered  to  trainees  after  completion  of  Individual  lessons, 
(3)  surveys  administered  periodically  to  OJT  supervisors,  and  (4)  system  use 
logs.  Evaluation  of  the  production  component  Included  Interviews  with  MDC 
personnel  and  an  ATCBT  authoring  system  log  maintained  throughout  the 
evaluation  period.  The  results  of  the  evaluation  are  detailed  In  Section  2.0. 
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SECTION  2.0 


PROTOTYPE  EVALUATION 
RESULTS  AND  RECOMMENDATIONS 

In  order  to  present  the  results  and  recomnendatlons  of  the  prototype 
evaluation  in  a  concise,  meaningful  way,  this  section  Is  organized  by  the  three 
components  of  the  ATCBT  system:  delivery,  management,  and  production.  Each  of 
the  subsections  provides  a  description  of  the  specific  objectives  under 
evaluation  for  that  component.  Although  some  of  the  objectives  cut  across 
components  (I.e.,  system  reliability  and  maintainability),  they  are  discussed 
under  the  component  deemed  most  appropriate.  To  facilitate  the  presentation  of 
specific  evaluation  results,  the  standardized  format  shown  below  was  used  In 
each  subsection: 

0  ISSUE:  Used  to  break  down  the  specific  evaluation  objectives  Into 
Issues  of  concern. 

0  RESULTS:  Used  to  document  activities  performed,  data  collected, 
problems  encountered,  lessons  learned,  etc. 

0  CONCLUSIONS/RECOMMENDATIONS:  Used  to  state  conclusions  that  can  be 
drawn  from  the  results  and  to  provide  recommendations  for  command¬ 
wide  Implementation. 

2.1  Delivery  Component 

A  primary  purpose  of  the  prototype  system  evaluation  was  to  determine  the 
effectiveness  of  the  ATCBT  system  In  delivering  training  to  the  local  work 
stations.  The  prototype  system  was  designed  to  provide  a  training  environment 
which  (1)  minimizes  time  away  from  the  Job,  (2)  makes  training  easily 
accessible  to  the  person  to  be  trained,  and  (3)  presents  Interactive,  self- 
paced,  performance-based  Instruction.  Information  on  the  effectiveness  of 
using  CBT  to  deliver  OJT  Instruction  at  the  local  workstations  was  collected 
from  the  users  In  the  aerial  ports.  There  are  three  primary  users: 

1.  The  trainees  who  are  trying  to  Increase  their  proficiencies  In 
relevant  STS/JQS  Items. 

2.  The  OJT/sectlon  supervisors  who  are  responsible  for  Identifying  the 
OJT  requirements  of  personnel  and  setting  proficiency  requirements. 

3.  The  ULC  monitors  who  are  responsible  for  supporting  the  OJT  training 
media  In  the  field. 

The  specific  objectives  for  evaluating  the  prototype  delivery  component 
were  defined  as  follows: 

1.  Was  the  CBT  Instruction  effective?  Did  the  CBT  lessons  provide  Job¬ 
relevant  training? 

2.  Did  the  CBT  lessons  show  the  potential  for  minimizing  supervisor  time 
required  to  train  personnel? 
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3.  How  were  the  CBT  lessons  utilized?  Which  lessons  were  completed? 

How  much  time  was  taken? 

4.  How  were  the  CBT  delivery  stations  utilized?  Were  they  accessible? 

Were  they  used  during  the  shift? 

5.  Were  the  ATCBT  support  requirements  sufficient? 

6.  Was  the  prototype  system  configuration  adequate? 

The  results  of  the  evaluation  are  organized  into  five  categories  in  the 
following  subsections:  (1)  Training  Effectiveness,  (2)  CBT  Lesson  Utilization, 
(3)  CBT  Station  Utilization,  (4)  CBT  Delivery  Support  Requirements,  and  (5) 
Prototype  Delivery  Media  Configuration. 

The  sources  of  data  used  for  the  evaluation  were: 

1.  Lesson  Performance  Data.  This  data  was  collected  at  all  three  sites 
while  the  lessons  were  being  taken.  Each  site  converted  the  data  to  a  format 
compatible  with  DDN  mail  transmission  and  sent  it  to  the  account  set  up  for 
this  purpose.  Hardware,  software,  and  procedural  problems  which  occurred 
primarily  at  the  onset  of  the  test  period  account  for  some  missing  data.  Data 
was  collected  at  all  sites  primarily  between  October  1985  and  February  1986 
after  nrast  problems  were  corrected.  ISD,  Inc.  received  raw  data  from  447 
lesson  executions  of  all  nine  lessons.  About  40X  of  the  lesson  data  was 
generated  by  AFSC  60530  students;  the  remainder  was  from  60531  students.  This 
data  does  not  include  all  people  who  took  lessons. 

2.  Lesson  Report  Surveys.  The  lesson  report  questionnaire  was  designed 
to  collect  written  feedback  from  the  target  population  of  trainees;  those  in 
transition  from  the  3-  to  5-skill  level.  This  information  describes  reactions 
to  the  materials,  system,  and  learning  environment.  The  evaluation  design 
called  for  each  person  taking  a  lesson  to  fill  out  a  questionnaire  following 
lesson  completion.  The  questionnaires  were  left  at  the  stations  and  completed 
on  a  volunteer  basis.  The  evaluation  period  ran  from  July  1985  to  February 
1986.  Lesson  reports  were  sent  to  ISO,  Inc.  by  ULC  monitors  with  dates  ranging 
from  August  1985  to  early  January  1986.  A  total  of  203  lesson  report  surveys 
were  included  in  the  analysis.  The  data  from  the  lesson  reports  is  summarized 
in  Table  2-1. 

3.  ULC  Monitor  Surveys.  At  the  conclusion  of  the  evaluation  period,  a 
debrief,  using  a  structured  questionnaire,  was  held  with  each  ULC  monitor  by 
the  ISO,  Inc.  project  manager.  The  questions  covered  (1)  effectiveness  of  CBT, 
(2)  data  management  usefulness  and  further  needs,  (3)  maintenance,  (4) 
training,  (5)  job  aids  needed,  and  (6)  training  support  personnel.  Many  of  the 
comments  related  to  the  delivery  component,  especially  in  the  first  category— 
CBT  effectiveness. 

4.  System  Logs.  ISO,  Inc.  kept  a  log  of  system  hardware  and  software 
problems  reported  by  the  ULC  monitors.  This  information  provides  a  picture  of 
what  might  be  expected  in  the  future. 


Table  2-1 


Lesson  Report  Item  Analysis 

NOTE:  Trainee  percentages  are  based  on  the  number  of  responses  per  location. 

The  combined  score  represents  percentages  based  on  the  total  number  of  trainee 
responses.  Supervisor  responses  were  conributed  from  Charleston  AFB  only. 

1.  Were  there  any  problems  during  the  lesson  (such  as  Interruptions,  system 
problems,  procedures  not  understood,  etc.)? 


Trainees 

Yes 

No 

No  Response 

Charleston 

6% 

94% 

0% 

Dyess 

n% 

89% 

0% 

Combi ned 

9t 

91% 

0% 

Supervisors 

19X 

80% 

0% 

2.  Were  you 

able  to  use  the 

system  when  you  wanted  to,  or 

did  you  have  to 

wait  to  come  back? 

Trainees 

Use  Immediately 

Walt 

No  Response 

Charleston 

82t 

18% 

0% 

Dyess 

91% 

9% 

0% 

Combined 

87% 

13% 

0% 

Supervisors 

90% 

10% 

0% 

3.  How  much 

time  did  you  have 

to  take  the  lesson? 

Average  Time 
(Measured) 

Adequate/Un 1 1m1 ted 

Trainees 

Time 

No  Response 

Charleston 

50%  •  2.8  hours 

47% 

3% 

Dyess 

28%  -  2,0  hours 

72% 

0% 

Combi ned 

39%  -  2.4  hours 

60% 

1% 

Supervisors 

32%  -  1.8  hours 

68% 

0% 

4.  Was  time 

taken  during  your 

shift  of  duty? 

Trainees 

Yes 

No 

No  Response 

Charleston 

90% 

10% 

0% 

Dyess 

56% 

44% 

0% 

Combi ned 

73% 

?7% 

0% 

Supervisors 

96% 

4% 

0% 

5.  Are  there 

other  lessons  In 

this  area  that  you  would  like 

to  see? 

Trainees 

Response 

No  Response 

Charleston 

29% 

71% 

Dyess 

3% 

97% 

Combined 

16% 

84% 

Supervisors 

46% 

54% 
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Table  2-1  (Continued) 


6.  Did  you 

take  this  lesson  to  advance 

in  your 

career? 

Trainees 

Yes 

No 

No  Response 

Charleston 

67% 

301 

31 

Dyess 

66% 

341 

01 

Combined 

66% 

321 

21 

Supervisors 

68% 

311 

11 

7.  Did  your  supervisor  assign  the 

lesson  or  did  you  decide  to  take  it 

yourself? 

Trainees 

Supervi sor-Ass i gned 

Self-Motivated 

No  Response 

Charleston 

381 

571 

51 

Dyess 

661 

341 

01 

Combined 

521 

461 

21 

Supervisors 

221 

761 

21 

8.  Do  you 

think  that  the  lesson  material 

is  relevant 

to  your  career 

advancement? 

Trainees 

Yes 

No 

No  Response 

Charleston 

851 

91 

61 

Dyess 

861 

141 

01 

Combined 

861 

121 

31 

Supervisors 

851 

121 

31 

9.  Do  you 

like  this  way  of  training? 

Trainees 

Yes 

No 

No  Response 

Charleston 

941 

31 

31 

Dyess 

751 

251 

01 

Combined 

851 

141 

11 

Supervisors 

911 

61 

31 

10.  Do  you 

have  any  suggestions  for  making 

this  lesson 

better  (such  as 

questions  asked  during  the  Isson  or 

the  way 

it  is  presented  or  tested)? 

Trainees 

Yes 

No 

No  Response 

Charleston 

161 

741 

101 

Dyess 

61 

891 

51 

Combi ned 

111 

821 

71 

Supervi sors 

211 

741 

51 
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2.1.1  Training  Effectiveness 

ATCBT  training  effectiveness  was  measured  by  (1)  whether  or  not  people 
learn,  (2)  how  job  related  the  learning  is,  (3)  interest  in  having  other  lesson 
topics  on  CBT  (as  a  sign  of  acceptance),  and  (4)  the  potential  of  CBT  to 
minimize  time  required  of  supervisors  for  training  605XX  AFSC  personnel. 

ISSUE: 

The  most  important  question  for  a  training  system  is,  "Does  the  intended 
target  population  learn  from  the  lessons?"  Were  the  aerial  port  personnel  able 
to  learn  effectively  with  ATCBT? 

RESULTS: 

0  Mastery  of  Lesson  Material 

To  determine  how  effective  the  training  was  for  learning,  the  segment 
test  items  were  analyzed  across  lessons  in  the  AFSC.  The  percent  of 
correct  (indicating  mastery)  versus  incorrect  (non-mastery)  test 
items  was  determined.  The  "percent  of  lesson  mastery"  was  computed 
by  dividing  the  total  number  of  correct  responses  by  the  total  number 
of  responses.  Table  2-2  shows  the  results  by  the  lessons  pertinent 
to  each  AFSC  and  then  by  each  of  the  three  sites.  An  overall 
percent,  by  lessons  in  the  AFSC,  is  also  shown  for  comparison. 


Table  2-2 

Group  Levels  of  Competency 


AFSC 

Lessons 

Available 

Percent  of 
Mastery  per  Lessons 

Average  Number  of 
Related  STS  Items 

60530 

3 

Charleston 

84t 

3.00 

Dyess 

62X 

6.85 

Rhein  Main 

SOX 

6.25 

Overall 

84X 

5.28 

60531 

6 

Charleston 

80X 

3.67 

Dyess 

82X 

7.53 

Rhein  Main 

8iX 

4.00 

Overall 

81X 

5.02 

For  the  three  60530  lessons,  the  average  mastery  was  84t.  A  percent 
of  68X  or  greater  should  be  expected  for  valid  items.  It  should  be 
noted  that  Oyess  AFB  results  show  only  62%.  However,  this  figure  Is 
acceptable  because  there  are  no  AFSC  60530  personnel  at  the  1st  MAPS 
at  Oyess  AFB  to  take  these  lessons.  The  lessons  were  taken  by  AFSC 
60531  personnel  who  are  not  familiar  with  either  the  manual  or 
automated  passenger  check-in  system.  They  should  reach  mastery  with 
additional  practice  on  the  lessons. 

The  six  lessons  related  to  60531  STS  Items  also  show  an  overall 
percent  mastery  of  slightly  better  than  80S.  In  all  cases,  complete 
mastery  should  be  reached  by  re-taking  those  parts  of  the  lesson  not 
satisfied. 

0  Job  Relatedness 

Table  2-2  also  shows  the  average  number  of  STS-related  knowledges 
which  those  people  In  the  data  sample  satisfied  by  mastery  of  Items 
tested.  Each  learning  objective  In  a  lesson  was  specifically  related 
to  STS  Items  during  the  design  of  the  lesson.  While  mastery  of  the 
CBT  lesson  material  does  not.  In  most  cases,  provide  full  proficiency 
on  STS  Items,  It  does  provide  knowledge  and  skills  leading  to  the 
full  STS  proficiency  required.  Full  STS/JQS  proficiency  usually 
requires  more  hands-on  training  than  CBT  allows. 

Another  Indication  of  job  relatedness  can  be  drawn  from  the  users. 
As  shown  In  Table  2-1.  the  lesson  reports  Indicated  that  the  majority 
of  trainees  (86t),  as  well  as  supervisors  (85X),  felt  that  the  lesson 
material  was  relevant  to  career  advancement  within  the  AFSCs.  ULC 
monitors.  In  the  final  debrief,  also  agreed  that  the  ATCBT  lessons 
were  highly  job  relevant;  much  more  so  than  the  slide/tape  programs. 

0  Acceptance  of  ATCBT 

Eighty-five  percent  of  the  trainees  Indicated  that  they  liked  the 
method  of  training.  Nineteen  percent  more  Charleston  trainees  than 
Oyess  trainees  responded  positively.  Ninety-one  percent  of  the 
supervisors  favored  the  use  of  CBT  for  Instruction. 

Another  Indication  of  acceptance  are  the  connents  from  lesson  reports 
on  other  areas  of  Instruction  the  participants  would  like  covered  by 
CBT.  Topics  named  as  desirable  for  CBT  training  Included  Form  109* s, 
all  cargo  handling,  MAC  regulations,  PSC,  PAX  dispatch,  ramp  service, 
tie  down,  vehicle  operations,  lost  and  found  and  many  broad  topics  In 
both  AFSCs.  Similarly,  supervisors  suggested  about  30  broad  topic 
areas  Including  statements  about  covering  5-sk111  level  topics,  "all 
ISDs,”  mobility,  phase  II,  and  other  advanced  training. 

CONCLUSIONS/RECOMMENDAIONS: 

Both  the  performance  data  and  perception  of  trainees  and  supervisors 
Indicated  that  (1)  learning  occurred,  and  (2)  the  learning  was  job  related 
(tied  to  STS/JQS  Items)  and  relevant  to  career  advancement,  and  (3)  CBT,  as  a 
method  of  training,  was  highly  acceptable. 
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There  are  no  direct  data  available  with  regard  to  how  ATCBT  might  affect 
supervisor  time  for  training  personnel.  Since  the  system  Is  training  effective 
and  does  not  require  supervisors  to  be  present  during  the  taking  of  a  lesson. 
It  should  be  assumed  that  ATCBT  will  not  add  to  time  required.  In  fact.  It  Is 
likely  to  decrease  supervisor  time  for  training  If  enough  lessons  are  available 
to  support  STS/JQS  proficiency  attainment.  The  need  expanding  the  lesson 
Inventory  has  already  been  considered  In  planning  for  command-wide 

Implementation.  Section  3.4.3  presents  the  CBT  lesson  development  list 

prioritized  by  need  In  the  field. 

2.1.2  Lesson  Utilization 

Management  data  reports,  as  well  as  subjective  feedback,  were  reviewed  to 
determine  (1)  how  lessons  were  utilized  and  (2)  what  motivated  people  to  take 
them.  Lessons  were  designed  to  meet  the  requirement  for  a  learning  segment 
that  was  short  enough  to  be  completed  between  work  tasks  and  other 

Interruptions.  A  primary  objective  was  to  facilitate  training  at  the  work  site 
which  would  not  conflict  with  the  normal  flow  of  duties.  Users  could  exit 

programs  after  completing  a  lesson  segment  and  return  at  a  later  time  to 
complete  other  segments. 

2. 1.2.1  Use  of  Lesson  Exam  Pretests 

ISSUE: 

Old  students  use  the  lesson  exams  as  pretests? 

RESULTS: 

The  student  responses  from  287  lessons  are  summarized  below. 


Pretests-  Pretests 
Used  Not  Used 

Charleston  57X  43X 
Dyess  SIX  49X 
Rhein  Main  49X  SIX 


Students  were  encouraged  to  take  lesson  exams  prior  to  going  through  the 
Instruction.  This  pre-test  capability  allowed  the  trainees  to  concentrate  on 
their  deficient  areas  without  wasting  time  on  previously  mastered  materials. 

Approximately  half  of  the  students  appeared  to  use  the  pretesting 
capability  built  In  through  the  lesson  exams.  Data  Is  Insufficient  to  form 
precise  correlations  between  pretesting  and  lesson  segment  selection,  but  some 
evidence  of  self-prescription  Is  shown  by  the  results  In  Figure  2-1. 

CONCLUSIONS/RECOMMENDATIONS: 

Lesson  exams  were  used  as  pretesters  by  some  students.  Clear  Instructions 
for  using  this  capability  must  be  provided  to  all  students  at  the  onset  of 
computer-based  training. 
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2. 1.2.2  Use  of  Pretesting  to  Select  Relevant  Training 

ISSUE: 

Was  differential  use  of  segments  demontrated? 

RESULTS: 

The  graphs  in  Figure  2-1  demonstrate  that  a  significant  number  of  trainees 
elected  to  pre-test  to  determine  their  knowledge  of  lesson  segment  contents. 
Low  numbers  of  users  on  individual  segments  are  one  indication  that  people 
tested  out  before  taking  the  lesson.  High  numbers  of  users  indicated  greater 
participation  in  lesson  review,  which  may  be  accounted  for  in  segments  that 
contained  unfamiliar  materials.  Blank  areas  appear  on  the  graphics  for  those 
lessons  containing  less  than  eight  segments;  they  do  not  imply  missing  data. 

Furthermore,  system  data  analysis  revealed  that  not  all  segments  were 
taken  in  order.  The  implication  is  that  some  discriminations  were  made  based 
on  utilization  of  pretesting  capabilities. 

CONCLUSIONS/RECOMMENDATIONS: 

The  pretest  capabilities  were  used  by  trainees  as  intended  to  afford 
flexibility  and  efficiency  in  segment  selection.  Training  time  was  spent 
effectively  since  effort  was  not  spent  going  over  materials  already  mastered  by 
the  users. 

2. 1.2. 3  Ratio  of  Initiation  to  Completion  of  Lessons 

ISSUE: 

How  many  learners  completed  lessons  once  they  initiated  them? 

RESULTS: 

Table  2-3  illustrates  the  number  of  lessons  initiated  by  users  and  the 
completion  results.  The  initiated  lessons  are  determined  by  the  total  sign-ons 
to  a  lesson  regardless  of  the  intent  of  the  user.  Incidental  sign-ons  were 
logged  onto  the  system  during  personnel  training  sessions  for  operation  of  the 
system.  These  are  indistinguishable  from  actual  lesson  user  sign-ons  and  are, 
therefore,  not  excluded  from  the  totals. 
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Figure  2-1.  Segment  Selection  Graph 
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Table  2-3 

Lessons  Initiated  vs.  Completed 


AFSC 

Total  Lessons 
Initiated 

Percent 

Completed 

Passenger 

60530 

Cargo 

60531 

Avg.  Completions 
Per  Trainee 

Charleston 

207 

25 

1.20 

1.37 

1.33 

Oyess 

108 

77 

2.20 

3.26 

3.08 

Rhein  Main 

137 

74 

3.33 

2.18 

2.36 

CONCLUSIONS/RECOMMENDATIONS: 

The  low  percent  of  completion  at  Charleston  reflected  a  large  number  of 
personnel  completing  Individual  lesson  segments  but  not  completing  the  lesson 
exam.  At  present,  lesson  completion  Is  determined  by  satisfactory  completion 
of  the  lesson  exam.  Tracking  Individual  lesson  segments  should  be  Incorporated 
Into  the  system  for  coomand-wlde  Implementation.  This  will  ensure  trainees 
receive  credit  for  completion  of  the  lesson  If  they  satisfy  the  segment 
requirements  defined  by  the  pretest  results. 

2. 1.2.4  Incentive  for  Training  on  CBT 

ISSUE: 

What  was  the  reason  for  lessons  being  taken? 

RESULTS; 

A  modest  majority  of  trainees  (67X)  Indicated  that  they  had  taken  the 
lessons  to  advance  their  careers.  It  Is  somewhat  surprising  that  almost  an 
equal  number  of  supervisors  (68X)  made  the  same  response.  This  may  Indicate 
the  need  for  refresher  and/or  cross-training  for  supervisors. 

Fifty-seven  percent  of  the  Charleston  trainees  elected  to  take  the 
lessons,  compared  to  only  34X  of  the  Oyess  trainees.  The  motivating  factor  at 
Dyess  appear  to  have  been  supervisor  assignment,  which  accounted  for  66t  of  the 
student  participation.  The  majority  (76S)  of  supervisors  taking  lessons 
elected  to  do  so,  although  almost  a  quarter  (22S)  were  assigned  to  the  lessons 
by  their  supervisors. 

CONCLUSIONS/RECOHMENOATIONS: 

Most  participants  took  the  lessons  to  advance  their  careers,  and  were 
self-motivated  to  take  them. 


2.1.3  CBT  Station  Utilization 

ISSUE: 

A  critical  Issue  Is  hOM  the  work  stations  were  utilized.  Utilization  Is 
examined  here  In  terms  of  system  accessibility,  use  of  shift  time,  time 
allotment  versus  actual  system  use,  and  impact  of  problems. 

RESULTS: 

0  Accessibility 

Work  stations  were  physically  accessible  to  most  of  the  target 
population  because  they  were  set  up  directly  at  or  nearby  the  actual 
Job  performance  sites.  Users  did  not  have  to  travel  or  suffer  major 
Interruptions  to  their  normal  work  day  routines  In  order  to  take  the 
lessons.  There  were  a  number  of  potential  users  at  Charleston 
International  Airport  and  at  Rhein  Main  who  were  physically  too  far 
removed  from  workstations  to  participate  In  lesson  testing.  Eighty- 
seven  percent  of  the  trainees  and  90X  of  the  supervisors  stated  that 
the  system  was  available  for  use  when  needed.  Charleston  users  had 
slightly  more  delay  time  than  Dyess  users. 

0  Use  of  Shift  Time 

Ninety  percent  of  the  Charleston  trainees  used  shift  time  to  take 
lessons.  The  somewhat  limited  hours  (0800-1700)  of  system 
availability  at  Dyess  may  account  for  only  56i  shift  time  use  by  the 
trainees.  Ninety-six  percent  of  the  supervisors  took  lessons  during 
shift  time. 

As  shown  In  Figure  2-2,  system  use  varied  with  the  type  of  lessons 
being  taken  as  well  as  the  hour  of  day.  Passenger  service  lesson  use 
was  heaviest  between  0800  and  1800  at  Dyess  and  Charleston,  but  at 
Rhein  Main,  It  was  much  more  pronounced  between  2400  and  0100.  Cargo 
handling  lessons  were  most  often  taken  between  0800  and  1700.  These 
use  patterns  seem  to  correlate  with  the  known  work  patterns  of  the 
two  groups  of  participants.  That  Is  to  say,  the  heaviest  lesson  use 
corresponds  with  the  lightest  workload  times. 

The  frequency  graph  numbers  are  based  on  actual  lesson  executions  (N 
»  316).  Invalid  frequency  data,  such  as  repeated  re-booting  without 
date  and  time,  was  distinguished  from  real  data  by  a  default-assigned 
date  of  January  1980.  The  latter  did  not  allow  assignment  to  real 
times  or  shifts  and  were  not  used. 
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Figure  2-2.  System  Use  Frequency  Graph 
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0  Time  Allotment  versus  Actual  System  Use 

The  majority  of  trainees  (601()  stated  that  they  were  given  adequate 
or  unlimited  time  to  take  lessons.  Those  who  were  given  blocks  of 
measured  time  (39t)  averaged  2.2  hours  on  the  system.  Sixty-eight 
percent  of  the  supervisors  had  adequate  or  unlimited  time  to  take 
lessons,  and  32%  of  them  demonstrated  an  average  of  1.8  hours  of 
measured  time.  Actual  system  use  showed  that  the  average  viewing 
time  per  lessons  was  60  minutes  for  the  60530  (passenger)  lessons, 
and  35  minutes  for  the  60531  (cargo)  lessons.  The  overall  average 
viewing  time  for  combined  lessons  was  42  minutes. 

0  Impact  of  In-Use  Problems 

The  majority  of  trainees  (92t),  as  well  as  supervisors  (Bit),  did  not 
experience  problems  while  running  the  lessons.  Among  those  who  did 
have  difficulties,  trainees  noted  system  problems,  while  supervisors 
noted  Interruptions  by  others  as  the  primary  sources  of  disruption. 
The  system  problems  noted  by  the  trainees  were  corrected  early  In  the 
evaluation  to  avoid  confounding  the  validation  process. 

CONCLUSIONS/RECOMCNDATIONS: 

User  accessibility  Is  good  among  the  currently  Installed  systems,  and  only 
a  minimum  number  of  problems  were  experienced  during  lesson  execution.  This 
results  In  user  perceptions  of  effective  training.  There  Is  a  need  to  Identify 
pockets  of  learners  In  remote  areas  to  enhance  accessibility  as  more  systems 
are  Installed  In  the  future. 

Most  users  were  given  adequate  time  during  their  work  shifts  to  complete 
lessons.  Although  the  lessons  are  designed  In  segments  which  can  be  completed 
In  1C  to  15  minutes  to  allow  for  workshift  Interruptions,  It  appears  that  most 
users  were  given  sufficient  time  to  ccxnplete  entire  lessons  (one  hour)  during 
slack  periods. 

A  calendar  clock  should  be  Installed  In  the  computer  hardware  to  assist  In 
accurate  data  entry  compilation. 

2.1.4  CBT  Delivery  Support  Requirements 

Interviews  with  the  ULC  monitors  and  OJT  supervisors,  and  comments  from 
students,  all  pointed  to  the  need  for  more  explicit  Information  at  the  CBT 
stations.  This  Information  Indues  (1)  CBT  student  job  aids,  (2)  adjunct 
training  materials,  and  (3)  supervisor  job  aids. 

2. 1.4.1  Terminal  Operation  Job  Aids 

ISSUE: 

How  much  does  a  student  need  to  know  In  order  to  use  the  CBT  equipment? 

RESULTS: 

The  ATCBT  system  was  designed  to  make  Interaction  extremely  easy. 
However,  there  were  still  Instances  In  which  students  were  not  aware  of  the 


operating  conventions.  For  example,  (1)  some  could  not  log  onto  a  lesson 

(I.e.,  type  In  six  digit  student  I.O.),  (2)  did  not  know  to  press  "RETURN*  when 

entering  a  selection  from  a  menu,  (3)  did  not  know  to  use  the  backspace  to 

correct  wrong  entries,  and  (4)  did  not  know  about  the  "ESC"  (escape)  key  to 
exit  from  a  lesson  Immediately  If  necessary.  Comments  from  students  and 
supervisors  alike  requested  specific  aids  on  basic  operation,  despite  the 

presence  of  typed  Instruction  sheets  In  notebooks  at  most  of  the  prototype 
sites. 

CONCLUSIONS/RECOHMENDATIONS: 

The  students  would  benefit  from  simplified,  but  specific  operating 
Information  (e.g.,  student  I.D.s)  and  keyboard  Instructions  posted  In  a 
prominent  area  (perhaps  even  stickers  on  the  machine).  This  would  reduce  delay 
time  In  the  student  deciding  what  to  do,  decrease  Inaccurate  entries,  and 
minimize  student  frustration. 

2. 1.4. 2  Adjunct  Materials 

ISSUE: 

Were  the  required  adjunct  training  materials  used  during  the  lessons? 

RESULTS; 

There  were  four  prototype  lessons  that  required  the  use  of  adjunct 
materials.  The  materials  Included  handouts  for  the  three  PACS  lessons  and  APR 
71-4  for  the  Hazardous  Materials  lesson.  Evidently,  none  of  these  materials 
were  made  available  at  the  work  stations.  Lack  of  the  PACS  handouts  probably 
accounts  for  most  of  the  Incorrect  responses  noted  In  examining  the  test  Item 
data. 

CONCLUSIONS/RECOMMENDATIONS : 

If  a  CBT  lesson  Is  built  around  the  use  of  adjunct  training  materials, 
then  those  materials  need  to  be  available  at  the  delivery  station.  This  can  be 
accomplished  by  making  the  statement  about  required  materials  that  occurs 
within  the  lesson  more  specific  and/or  stronly  worded  and  by  developing  a  list 
of  required  materials,  so  the  ULC  can  establish  a  "library"  of  materials. 
Responsibility  should  lie  with  the  OUT  supervisor,  but  assistance  should  be 
given  as  needed. 

2. 1.4. 3  Supervisor  Job  Aids 

ISSUE: 

What  tools  and  resources  does  the  OJT  supervisor  need  In  order  to 
facilitate  his  training  responsibilities  and  Increase  the  effectiveness  of  CBT? 

RESULTS: 

Many  supervisors  expressed  a  need  to  be  more  familiar  with  the  content  of 
the  CBT  lessons.  Lesson  specifications  were  Included  In  the  ATCBT  User  Guide 
to  outline  the  learning  objectives,  content,  and  evaluation  strategies  for  each 
prototype  lesson.  Evidently,  these  were  either  not  available  or  not  used  by 
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the  supervisors.  A  catalog  of  CBT  lessons  Is  suggested.  This  might  already  be 
available  in  the  form  of  the  Transportation  Training  Catalog  described  in 
Section  2.3.4.  Hovfever.  the  relation  bettmen  the  available  lessons  and  OJT 
requirements  is  not  described  in  the  catalog.  Supervisors  need  a  tool  to 
preview  lessons  before  making  assignments  to  students.  As  the  number  of 
Ussons  grows,  the  opportunity  to  take  each  lesson  may  not  be  possible. 

CONCLUSIONS/RECOMMENDATIONS: 

Supervisors  need  to  have  specific  lesson  summary  information  in  order  to 
help  their  decision  making  in  utilizing  CBT  to  supplement  OJT.  This  summary 
should  not  be  limited  to  CBT  lesson  titles,  as  currently  included  in  the 
Training  Program  Catalog.  Specific  references  to  STS  items  need  to  be  made  in 
order  to  provide  information  to  the  OJT  supervisor  as  to  which  CBT  lessons 
relate  to  which  JQS  items.  Thus,  the  supervisor  can  more  effectively  and 
efficiently  assign  CBT  lessons  to  supplement  OJT  requirements. 

2.1.5  Prototype  Delivery  Configuration 

The  prototype  system  design  considered  both  the  screen  resolution  and 
storage  required  to  deliver  instruction  to  the  student  effectively. 

ISSUE: 

Is  the  ATCBT  system  adequate,  as  configured,  for  delivering  instruction? 

RESULTS: 

Based  on  student  and  supervisor  comments  at  the  prototype  sites,  the 
graphics  presentation  and  resolution,  as  delivered  in  the  lessons,  was  well 
received.  There  was  a  call  for  more  illustrations  and  flow  charts,  but  not  for 
improvement  in  quality. 

The  current  delivery  station  configuration  consists  of  a  10  megabyte  hard 
disk  for  lesson  storage.  The  hard  disk  was  originally  included  to  preclude 
long  delays  in  downloading  lessons  from  the  ULC  to  the  delivery  stations. 
However,  this  configuration  was  only  tested  for  the  initial  nine  lessons.  The 
first  lessons  consumed  773K  of  memory,  including  all  text  files  and  picture 
files.  Given  an  average  of  approximately  86K  per  lesson,  only  about  116 
lessons  would  be  able  to  fit  on  a  10  megabyte  hard  disk,  excluding  the  run-time 
version  of  COS  and  any  locally  generated  student  data.  Since  60  lessons  (over 
half  of  the  disk  capacity)  will  already  be  in  the  field  by  1  October  1986,  with 
a  total  of  over  200  lessons  by  1988.  the  need  for  expanded  storage  is  apparent. 
A  21  megabyte  disk  could  contain  about  230  lessons  with  run-time  software, 
which  would  be  sufficient  for  the  200  lessons  planned. 

CONCLUSIONS/RECOMMENDATIONS: 

The  delivery  station  configuration  should  include  a  21  megabyte  hard  disk. 
There  is  no  requirement  for  faster  processing  or  increased  screen  resolution 
during  lesson  delivery. 
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2.2  Management  Component 

The  original  concept  of  the  management  component,  as  cefined  by  the 
contract  Statement  of  Work,  was  that  It  consisted  of  information  management. 
That  Is,  the  management  component  would  provide  consolidated  and  summarized 
training  program  Information  to  personnel  at  the  base  level,  as  well  as  higher 
headquarters  such  as  Numbered  Air  Forces  and  HQ  MAC  at  Scott  AFB.  While  this 
original  management  function  remains,  other  management  functions  have  been 
added  to  the  concept  of  the  ATCBT  management  component.  These  additions  are  a 
result  of  what  was  learned  during  planning.  Implementing,  and  evaluating  the 
prototype.  In  order  to  organize  the  discussion  to  follow,  a  description  of 
the  complete  management  component  Is  provided  below. 

To  begin  the  description.  It  Is  relevant  to  make  a  general  statement  about 
what  the  word  "management"  has  now  come  to  mean  within  the  context  of  the  ATCBT 
system.  ATCBT  management  means  the  methods  by  which  the  ATCBT  organization 
plans,  operates,  and  controls  resources  to  meet  Its  goals  and  objectives.  By 
methods,  we  mean  Inputs,  process,  and  outputs.  It  Is  the  methods  which  are  the 
primary  focus  of  the  prototype  evaluation.  Figure  2-3  represents  the  generic 
elements  of  the  ATCBT  management  method. 


INPUTS 
REQUIRED 

Figure  2-3.  Model  of  ATCBT  Management  Methods 


OUTPUTS 

>  PROCESS  . >  NEEDED 


The  ATCBT  resources  are  not  only  (1)  Information,  but  (2)  lessons,  (3) 
computer  hardware  and  software,  (4)  facilities  such  as  communication  lines  and 
space  for  terminals,  and  (5)  the  people  In  the  organization,  including 
OJT/sectlon  supervisors  and  ULC  monitors.  Figure  2-4  represents  the  resources 
of  ATCBT. 


GOALS 

Consolidate  and  Summarize 
TRAINING  INFORMATION 
for  Access  by  Managers 

Provide  Training,  Job  Aids,  and 
Workable  Procedures  to  the 
TRAINING  ORGANIZATION 

Update,  Control,  and  Disseminate 
LESSONS 

Operate  and  Maintain 
EQUIPMENT  AND  SOFTWARE 

Prepare  and  Maintain 
FACILITIES 

Figure  2-4.  Reaching  Goals  by  Management  of  ATCBT  Resources 
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The  general  ATCBT  goals  related  to  these  resources  are: 

0  Provide  the  appropriate  ATCBT  training  Information  to  managers  and 
users  at  the  tW  It  Is  needed.  Key  words  here  are  "appropriate"  and 
"time." 

0  Ensure  that  all  people  In  the  organization  know  their 
responsibilities  and  have  the  training  and  support  to  accomplish 
their  tasks. 

0  Ensure  that  the  CBT  lesson  Inventory  Is  current  and  readily 
accessible  to  aerial  port  personnel. 

0  Ensure  that  all  equipment  and  software  Is  operational  and  maintained 
for  maximum  availability. 

0  Ensure  that  facilities  such  as  power,  communications,  and  space  are 
adequate  for  operations. 

The  remainder  of  this  section  Is  organized  by  the  five  categorless  of 
ATCBT  resources:  (1)  Training  Information  Management.  (2)  Organization 

Management.  (3)  Lesson  Management.  (4)  Equipment  and  Software  Management,  and 
(5)  Facilities  Management. 

2.2.1  Training  Information  Management 

The  methods  for  management  of  ATCBT  training  Information  are  depicted  in 
Figure  2-5  below. 


INPUTS  — > 


PROCESS  — >  OUTPUTS 


0  Trainee  register 

0  Lesson  catalog 

0  Trainee  performance 
data 


0  Input  trainee  and  o  Printed  reports 
lesson  data 

0  Queries 

0  Collect  performance 
data 

0  Generate  summaries 


Figure  2-5.  ATCBT  Information  Management  Methods 


In  designing  the  prototype,  decisions  on  who  needed  Information,  what 
Information  was  needed,  and  when  It  was  needed  It  were  made  as  a  result  of  a 
series  of  meetings  with  personnel  from  HQ  MAC  and  the  MDC.  These  meetings 
defined  the  outputs  as  a  series  of  printed  reports.  In  order  to  generate  the 
reports.  It  was  necessary  to  provide  a  way  to  Identify  the  trainees  and  the 
lessons  available.  Information  related  to  supervisors,  trainee  work  areas, 
recurring  training  requirements,  and  other  associated  data  also  needed  to  be 
included  In  the  database  In  order  to  manipulate  data  for  the  reports. 
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It  was  envisioned  that  the  ULC  would  be  the  focal  point  for  registering 
trainees,  collecting  all  trainee  data,  and  disseminating  management  Information 
at  the  base  level.  It  was  anticipated  that  the  ULC  would  then  send  the 
sunmarlzed  data,  via  the  Defense  Data  Network  (DDN),  to  an  electronic  mall  box 
which  would  be  accessed  by  the  HOC  for  further  dissemination.  As  the 
Information  controller  at  the  base,  the  ULC  was  responsible  for  maintaining  the 
Integrity  of  the  database.  Including  keeping  the  trainee  roster  updated  as 
personnel  entered  or  left  the  aerial  ports. 

For  the  purpose  of  the  prototype,  all  Information  dissemination  was 
reported  via  printed  reports.  The  consolidation,  summarization,  and  report 
generation  of  the  performance  data  was  Implemented  via  a  commercl ally-avail  able 
database  management  program  (dBase  II  from  Ashton-Tate).  By  using  a 
commercial ly-avall able  database  management  program.  It  was  possible  to  build  In 
a  flexible  data  management  system  which  could  be  easily  programmed  for 
differing  requirements.  It  was  anticipated  that  the  experience  of  the 
prototype  would  provide  more  detailed  knowledge  of  what  was  needed  In  the  field 
at  all  levels  of  management.  The  ULC  monitors  Interfaced  directly  with  the 
data  management  programs  via  a  series  of  prompts  and  menus. 

The  data  management  options  provided  eight  printed  reports.  These 
Included:  (1)  reports  to  both  the  supervisor  and  student  upon  successful 
completion  of  a  lesson  examination,  (2)  a  utilization  report  showing  the 
lessons  taken  during  a  given  period  and  the  number  of  people  taking  them,  (3)  a 
report  showing  detailed  and  exact  responses  made  during  a  lesson,  (4)  a  report 
showing  test  Item  statistics  across  students  during  a  given  period,  (5)  a  list 
of  lessons  available,  (6)  a  list  of  those  personnel  registered  to  take  lessons 
but  not  taking  any,  (7)  a  list  of  lessons  during  a  given  period  for  which  there 
was  no  activity,  and  (8)  an  exception  report  which  was  intended  to  show 
trainees  having  problems. 

The  evaluation  Issues,  results,  and  conclusions  below  were  derived  from  a 
set  of  survey  questionnaires  administered  to  the  ULC  monitors  at  the  prototype 
sites,  as  well  as  personnel  from  the  HOC  and  HQ  MAC.  In  the  discussions  with 
the  ULC  monitors.  It  was  requested  that  they  consider  comments  made  by  OJT 
supervisors  and  students  who  received  reports  or  requested  Information,  as  well 
as  their  own  observations  or  Ideas  as  to  what  was  useful.  Most  of  the  comments 
received  were  acquired  during  the  last  two  weeks  of  the  prototype  evaluation  In 
order  to  take  full  advantage  of  the  experience  gained.  The  descriptions  below 
are  a  composite  of  the  comments. 

2.2. 1.1  Information  Management  Procedures 

ISSUE: 

Were  the  ULC  procedures  for  Inputting  database  Information  and  acquiring 
performance  data  reports  adequately  explained  and  easily  used? 

RESULTS: 

0  User  Guide 

The  ATCBT  User  Guide,  which  explains  the  procedures  for  Inputting 
student  and  lesson  data  as  well  as  generating  reports,  was  modified 
twice  during  the  evaluation  to  reflect  the  need  to  clarify  and  expand 
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the  Instructions.  During  the  Installation  at  each  prototype  site,  a 
demonstration  with  hands-on  training  In  the  use  of  the  data 
management  procedures  was  provided.  In  all  cases,  this  seemed 
adequate  and  no  negative  comments  were  received. 

0  Inputting  Trainee  Registrations 

Suggestions  received  Included  providing  a  standard  procedure  and  form 
for  Inputting  Information  on  trainees  entering  Into  a  squadron  so 
they  could  be  entered  Into  the  system  as  quickly  as  possible.  The 
1st  MAPS  at  Dyess  AFB  Instituted  a  requirement  for  all  cargo 
personnel  entering  Into  the  squadron  at  3-  and  5-sk111  levels  to  take 
all  cargo  CBT  lessons.  This  was  done  to  refresh  personnel  In  various 
training  topics  which  were  available  In  the  prototype  lessons.  In 
the  future,  1st  MAPS  plans  to  continue  the  practice  In  some  form. 
This  means  new  entries  In  the  system  must  be  made  Immediately. 

0  Registration  Roster 

One  shortcoming  In  the  current  procedures  seemed  to  be  the  lack  of 
capability  to  generate  a  roster  of  those  people  registered  as 
trainees.  This  was  overcome  during  the  prototype  by  addition  of  a 
list  function  as  a  temporary  solution. 

0  Performance  History  Data 

An  area  which  had  never  been  addressed  In  Implementing  the  prototype 
was  how  long  the  performance  data  should  be  kept  at  the  ULC.  An 
associated  question  was  how  should  the  the  trainee's  performance  data 
be  sent  to  his  or  her  next  duty  station.  No  policy  or  procedures 
were  set  as  a  result  of  prototype  operational  practice  at  the  bases. 
Floppy  disk  copies  of  data  wer%  being  kept.  However,  since  the 
duration  of  the  evaluation  period  was  six  months  at  the  most,  the 
problem  of  how  long  to  keep  data  did  not  arise. 

0  Dispersal  of  Base  Management  Information 

The  trainee  registration  file  currently  Includes  Information  related 
to  the  trainee's  supervisor  and  work  center.  These  were  Intended  to 
allow  designation  of  where  a  report  should  go.  Variations  were 
observed  In  how  ULC  monitors  used  the  supervisor  field.  In  some 
cases,  the  level  of  supervisor  was  at  an  Immediate  supervision  level. 
In  other  cases,  the  supervisor  was  at  a  higher  level.  The  question 
was  raised  as  to  whether  the  trainee  reports  could  be  grouped  only  by 
work  center  as  opposed  to  supervisor.  This  would  allow  generation 
and  dispersal  of  reports  by  work  center  allowing  the  work  center  to 
split  up  the  reports  to  specific  supervisors.  A  primary  question  of 
concern  here  was  whether  It  was  possible  to  keep  the  supervisors 
updated  as  personnel  changed  work  areas  or  supervisors. 

CONCLUSIONS/RECOMMENDATIONS 

Revise  the  section  of  the  ATCBT  User  Guide  which  provides  Instructions  for 
data  management  to  show  examples  of  reports  and  procedural  Interactions  with 
the  data  management  routines  (menus  and  prompts).  Include  procedures  for  In- 


processing  of  new  personnel,  out- processing  of  personnel  leaving  the  squadron 
(Including  what  to  do  with  their  AiCBT  performance  records),  and  reflect  the 
changes  In  report  requirements  discussed  In  the  next  section.  Custom  screens, 
which  allow  easier  viewing  of  data,  should  be  considered. 

Establish  a  policy  for  how  long  the  ULC  should  keep  data  on  floppy  disks 
and  printouts.  It  Is  anticipated  that  the  ULC  will  be  the  only  depository  for 
the  data  since  no  requirements  have  been  Identified  on  a  regular  basis  for  data 
at  higher  headquarters  or  the  MDC.  Rather,  the  data  will  be  accessed  by 
request  from  these  other  activities.  It  Is  recommended  that  APR  12-50  be  the 
basis  for  establishing  the  history-keeping  policy.  According  to  Table  50-4  of 
the  regulation  (rule  15),  the  training  data  should  be  destroyed  "when 
superceded,  obsolete,  or  no  longer  needed,  whichever  Is  sooner."  It  Is 
recoomended  that,  when  a  trainee  leaves  the  port,  he  or  she  be  given  a  printout 
of  their  record  as  well  as  sending  the  record  over  the  DON  network  to  the  new 
duty  station  If  It  Is  on  the  ATCBT  system.  The  transmission  should  be  studied 
further  to  determine  Its  Impact  on  time  and  resources. 

2. 2. 1.2  Information  Management  Reports 

ISSUE: 

Were  the  reports  useful?  This  Issue  Includes  questions  related  to  what 
reports  are  needed,  when  are  they  needed,  how  should  they  be  distributed  and  to 
whom,  and  how  often  should  reports  be  generated? 

RESULTS: 

0  Frequency  of  Data  Summarization  and  Reporting 

Generally,  each  squadron  summarized  data  and  generated  reports  on  the 
basis  of  the  amount  of  activity  that  had  gone  on  In  the  preceding 
period  rather  than  a  specific  periodic  basis.  The  1st  MAPS,  for 
example,  had  periods  of  relative  ATCBT  Inactivity  when  personnel  were 
mobilized.  There  was  no  need  to  consolidate  the  data  during  these 
periods.  Likewise,  the  other  squadrons.  Charleston  APB  and  Rhein 
Main  AB,  had  times  when  activity  was  light  as  a  result  of  exercises 
or  other  demands  on  time.  Some  constraints  In  the  time  available  to 
the  ULC  monitor  to  generate  reports  was  also  evident.  The  time 
needed  to  consolidate  data  and  generate  reports  was  not  always 
Immediately  available  and  sometimes  data  waited  days  to  be  processed. 

0  Reporting  In-Progress  Training  Data  to  Supervisors 

There  was  almost  universal  agreement  on  the  need  to  change  the 
completion  reports  which  were  designed  to  be  distributed  to  both  the 
trainee  and  the  trainee's  supervisor.  All  units  Indicated  that  (1) 
only  one  report  was  needed  for  distribution  to  the  supervisor  (which 
would  still  be  Included  In  the  trainee's  623  folder),  and  (2)  the 
completion  reports  should  provide  a  complete  record  of  progress  by 
lesson  for  the  trainee.  Currently,  the  report  Is  only  generated  If 
the  lesson  examination  Is  taken  and  then  only  the  score  on  the  lesson 
examination  Is  provided.  Supervisors  Indicated  that  they  would  like 
to  know  what  progress  Is  being  made  on  a  lesson  regardless  of  whether 
or  not  the  lesson  exam  Is  completed.  In  other  words.  If  a  student 
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started  on  one  date  and  completed  one  segment  of  a  lesson  and  then 
returned  two  weeks  later  for  two  more  segments  and  had  not  returned 
for  completion  of  the  lesson  exam,  the  supervisor  would  like  to  know 
when  training  was  taken  and  what  score  the  trainee  received  on  each 
part  of  the  lesson.  This  would  allow  the  supervisor  to  know  the  pace 
of  progress  as  well  as  the  proficiency  In  specific  areas. 

0  Reporting  History  Training  Data  to  Supervisors 

All  sites  Identified  the  need  for  a  trainee  history  report.  This  Is 
currently  not  part  of  the  data  management  process.  The  history 
report  would  Include  all  lessons  or  parts  of  lessons  taken  by  a 
student  with  the  date  and  score.  This  data  should  be  kept 
permanently  as  long  as  the  trainee  Is  at  the  port  so  that  reports  can 
be  generated  at  the  request  of  the  supervisor  or  other  managers. 

0  On-Line  Queries  by  Supervisors 

Some  questions  were  raised  as  to  the  feasibllty  and  need  for  on-line 
access  by  supervisors  to  the  history  or  current  progress  of  an 
Individual  assigned  to  that  supervisor.  There  was  not  a  clear 
consensus  as  to  the  need  for  an  on-line  query  capability.  One  ULC 
monitor  suggested  that  It  could.  In  fact,  be  somewhat  dangerous  to 
allow  supervisors  to  have  access  to  data  at  a  terminal.  Although 
various  protection  schemes  could  be  Implemented  to  ensure  Integrity 
of  the  data,  the  Implementation  of  the  on-line  query  would  require 
modifications  to  the  design  of  the  CONTROL  and  comnunl cations 
programs,  as  well  as  the  data  management  programs  and  files. 

0  Other  Reports 

Another  report,  which  does  not  exist  now  and  was  Identified  by  all 
sites  as  being  needed.  Is  a  list  of  personnel  registered  on  the  ATCBT 
system.  This  was  also  mentioned  In  Section  2.2. 1.1.  This  oversight 
obviously  needs  to  be  corrected.  The  listing  of  lessons.  Including 
recurring  training  requirement  lessons,  was  considered  adequate, 
especially  as  the  number  of  lessons  grow.  The  CBT  utilization  report 
was  also  judged  to  be  adequate,  although  there  were  connents  as  to 
whether  all  data  was  Included  on  the  report.  This  probably  relates 
to  a  problem  with  the  algorithm  which  computes  viewing  time  by 
terminal. 

The  reports  which  present  the  detailed  response  by  Item  data  and  the 
test  Item  analysis  data  were  not  subject  to  comment  by  the  squadrons 
since  the  reports  are  not  for  their  use.  These  reports  are  for  the 
lesson  validation.  The  data  for  the  evaluation,  the  report  recording 
actual  Incorrect  responses.  Is  normally  collected  only  for 
developmental  periods.  However,  during  the  prototype,  all  data  was 
collected  and  analyzed  by  ISO,  Inc.  There  were  no  negative  comments 
on  these  reports  from  the  MDC.  However,  the  MDC  would  like  the 
capability  to  receive  supervisor  conments  on  lesson  content  as  part 
of  the  data  management  reporting  function,  rather  than  relying  on 
written  comments  getting  passed  to  the  ULC  and  then  sent  to  the  MDC 
via  DON  or  mall. 
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The  two  file  management  reports,  which  present  the  list  of  students 
not  showing  lesson  completions  and  the  lessons  not  showing  activity, 
were  not  judged  disfavorably.  However,  It  should  be  noted  that  the 
student  printouts  will  list  everyone  who  has  not  actually  completed  a 
lesson  regardless  of  whether  they  showed  some  activity.  Thus,  In  a 
squadron  with  600  people,  during  a  one-week  time  period,  many  people 
may  not  show  actIvUy  and  the  list  could  be  quite  long. 

CONCLUSIONS/RECOMMENOATIONS: 

The  completion  report  should  reflect  more  detailed  data  Including  segment 
exam  as  well  as  lesson  exam  scores  and  the  dates  that  any  activity  occurred  for 
each  student.  This  will  allow  the  supervisor  to  more  effectively  Judge  the 
rate  of  progress  and  proficiency  of  personnel.  This  will  also  allow  the 
supervisor  to  determine  that  adequate  time  Is  being  spent  In  training  that  has 
been  scheduled.  The  report  does  not  need  to  go  to  the  student. 

An  historical  file  system  for  each  student  should  be  Implemented.  This 
file  will  provide  a  report  to  contain  all  ATCBT  activity  as  long  as  the  trainee 
Is  In  the  squadron.  The  updated  historical  report  could  be  (1)  the  printout 
Included  In  the  person's  Individual  training  record,  (2)  the  report  which  could 
be  accessed  by  supervisors,  either  through  printouts  or  on-line  access,  and/or 
(3)  the  document  which  would  be  sent  to  the  person's  next  duty  station  after 
leaving  the  squadron. 

The  need  for  on-line  query  by  supervisors  at  the  work  station  terminals 
needs  to  be  reviewed  further.  Mildly  conflicting  Judgements  on  the  part  of  ULC 
monitors  and  other  managers  of  training  exist  on  this  feature.  It  Is  feasible 
to  provide  the  capability,  within  limits,  while  still  providing  protection  of 
data  files  and  their  access.  However,  the  exact  query  capability  needs  to  be 
defined  for  the  design  to  be  scoped.  Likewise,  the  requirements  to  allow 
supervisors  to  Input  lesson,  revision  conments  for  the  MDC  on-line  should  be 
examined  further.  Modifications  to  the  CONTROL  software  and  data  management 
files  are  anticipated  to  support  these  features.  The  CONTROL  software  handles 
all  tasking.  Including  background  processing  and  communications. 

The  description  of  the  data  management  process  and  procedures  In  the  ATCBT 
User  Guide  need  to  be  revised  to  reflect  changes  made  for  reports,  on-line 
queries,  or  other  changes  which  may  subsequently  be  approved. 

2.2.2  Organization  Management 

The  ATCBT  management  methods  related  to  organization  are  shown  In  Figure 
2-6  below. 

During  the  prototype  evaluation,  the  primary  concern  was  the  organization 
responsibilities  for  base  ATCBT  operations,  as  opposed  to  higher  headquarters 
such  as  Numbered  Air  Forces  or  HQ  MAC. 
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Figure  2-6.  ATC6T  Organization  Management  Methods 


As  seen  In  Figure  2-6,  the  organizational  elements  at  the  base  level  are 
the  Training  Sectlon/ULC,  OJT  and  section  supervisors,  several  squadron/base 
support  activities,  and  the  1492  ATTF  (with  particular  emphasis  on  the  MDC). 
These  groups  must  work  In  concert  to  ensure  correct  operations  and  maintenance 
of  the  ATCBT.  In  addition,  there  are  some  support  functions  required  from  a 
central  source  such  as  the  MDC  for  source  and  system  Information.  The  ATCBT 
organization  Issues  relate  to  what  responsibilities  each  activity  must  have. 

2.2.2. 1  Roles  of  the  Training  Sectlon/ULC 

ISSUE: 

What  roles  must  the  Training  Section  (the  ULC  monitor  In  particular)  play? 
For  these  roles,  do  the  skills  and  time  required  match  the  existing  people 
resources.  Implied  In  this  Is  that  no  additional  manning  Is  desired  to  support 
ATCBT  Implementation. 

RESULTS: 

0  ULC  Monitor's  Role  and  Responsibility 

The  primary  responsibility  for  the  ATCBT  system  at  each  site  rested 
on  the  ULC  monitor  and  whatever  staff  was  available  to  him.  The  ULC 
monitor  was  trained  to  operate  the  system,  perform  preventive 
maintenance,  run  diagnostics  for  corrective  maintenance  and  obtain 
corrective  actions.  Initiate  summarization  of  performance  data  and 
dissemination  of  results  In  written  form  to  the  section  supervisors; 
provide  the  proper  adjunct  materials  for  lessons;  provide  Information 
on  use  of  ATCBT  to  trainees  and  supervisors;  and  demonstrate  ATCBT  to 
Interested  parties. 

0  ULC  Monitor's  Background  and  ATCBT  Training 

Of  three  sites,  only  one  ULC  monitor  had  any  appreciable  experience 
with  microcomputers  of  any  sort.  Thus,  entry  level  Into  the  position 
was  In  reality  a  non-computer  user.  Two  monitors  were  AFSC  75172 
(Training)  and  one  was  AFSC  60572  (Air  Transportation).  In  the  final 


survey  of  ULC  monitors,  none  felt  that  the  skills  required  were  too 
difficult.  While  some  procedures  could  be  made  more  efficient,  none 
Mere  beyond  the  attainment  of  a  typical  ULC  monitor.  Specific 
comments  on  ease  of  procedures  and  needs  for  job  aids  are  r'^ve'^ed 
later  In  Section  2.2.4. 

0  ULC  Monitor's  Time  for  ATCBT  Duties 

While  the  ULC  monitors  received  the  ATCBT  concept  and  systems 
enthusiastically,  they  also  expressed  concern  over  time.  They  noted 
that  the  billet  calls  for  several  training  support  tasks  In  addition 
to  directing  the  ULCs.  Upgrade  training,  HHE  training,  filling 
formal  school  quotas,  and  administrative  paperwork/records  are  part 
of  the  job.  In  addition,  during  Installations  at  both  Charleston  AFB 
and  Rhein  Main  AB,  ULC  monitors  were  part  of  preparation  for 
exercise/inspections.  Both  of  these  squadrons  are  600-plus-people 
organizations  with  sizeable  training  commitments. 

CONCLUSIONS/RECOMMENDATIONS: 

Since  no  additional  manning  Is  desirable  or  possible,  the  ATCBT  procedures 
must  be  as  labor  efficient  as  possible.  The  success  of  ULC  monitors  to  date 
Indicates  that  their  skills  are  sufficient  to  learn  the  duties  required  for 
ATCBT.  The  user  documentation  training  and  support  must  maximize  the  monitor's 
time,  however,  since  It  Is  limited.  Continued  efforts  must  be  made  to  ensure 
that  all  of  the  tasks  required  of  the  ULC  monitor  are  made  more  efficient.  In 
addition,  there  are  support  functions  recommended  from  other  groups.  These 
functions  are  beyond  the  scope  of  the  ULC,  but  within  the  scope  of  the  other 
groups  (see  Section  2. 2. 2. 3). 

2. 2. 2. 2  Roles  of  OUT  and  Section  Supervisors 

ISSUE: 

What  roles  are  required  of  the  OUT  and  section  supervisors?  Does  ATCBT 
reduce  or  minimize  the  time  required  of  a  supervisor  for  training? 

RESULTS: 

0  Supervisor  Interest 

An  Indication  of  supervisor  Interest  In  ATCBT  Is  that  about  SOS  of 
the  people  taking  lessons  on  the  system  at  the  two  larger  aerial 
ports  were  5-  and  7-sk111  levels  Instead  of  the  targeted  3-sk111 
level.  According  to  the  ULCs,  there  was  no  doubt  that  part  of  the 
reason  was  that  the  supervisor  needed  to  know  what  was  there  In  order 
to  recommend  It  or  require  It  for  subordinates. 

0  Saving  Supervisor  Time 

It  Is  difficult  to  tell  how  much  time  ATCBT  saves  supervisors  In 
achieving  training  goals  because  the  nine  prototype  lessons  represent 
only  a  small  sample  of  training  needs.  However,  one  antecdote 
Illustrates  the  potential.  The  three  lessons  for  AFSC  60530  cover 
the  PACS  (Passenger  Automated  Check-In)  system  which  has  just 
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recently  been  Installed  at  Rhein  Main.  During  Installation,  a 
supervisor  at  Rhein  Main  Indicated  that  he  learned  the  "why"  of 

things  he  had  been  doing  for  several  months  and  that  the  lessons 
provided  some  knowledge  that  he  had  to  discover  for  himself.  He 
indicated  that  all  of  his  people  would  take  t;.d  lessons  as  soon  as 
possible.  During  the  same  week,  several  605X0  supervisors  from 
nearby  Ramsteln  AB  dropped  in  for  a  demonstration  and,  as  a  result. 
Indicated  that  they  would  be  sending  all  of  their  people  to  Rhein 
Main  to  take  the  three  lessons. 

0  Supervisor  Catalog  for  ATCBT 

The  need  for  supervisors  to  know  what  Is  available  points  to  a 

problem  which  occurred  at  times  during  the  evaluation.  Although 

demonstrations  were  provided  to  supervisors  and  Information  was  at 
the  ULCs  on  which  lessons  were  available,  no  "catalog"  of  lessons  was 
disseminated  at  any  site.  It  was  pointed  out  by  one  ULC  monitor  that 
a  catalog  of  lessons  by  work  center  would  be  useful  as  well  as  a 

general  catalog.  Summaries  of  lessons  available  were  suggested  for 
others,  such  as  Branch  Chiefs,  Senior  NCOs,  and  Commanders. 

0  HQ  MAC  Requirement  to  Use  ATCBT 

A  meeting  was  held  with  23  section  supervisors  and  the  OJT  supervisor 
at  Charleston  AFB  at  the  conclusion  of  the  evaluation  period.  This  meeting 
surfaced  concern  over  whether  HQ  MAC  would  make  any  of  the  training  mandatory. 
Supervisors  stated  that  they  wanted  the  freedom  to  make  these  decisions 
themselves. 

CONCLUSIONS/RECOMMENDATIONS: 

During  each  site  Implementation,  demonstrations  and  hands>on  experience 
should  be  provided  to  all  supervisors.  Periodic  demonstrations  should  be  held 
for  new  supervisors  until  all  are  familiar  with  the  system. 

Catalogs  should  be  developed  by  the  MDC  to  be  disseminated  by  the  ULC 
during  Implementation.  Updating  should  occur  with  new  lessons.  Catalogs 
should  describe  content,  objectives,  relationship  to  STSs,  prerequisites,  and 
other  general  Information  by  which  supervisors  can  select  training  to  meet  an 
Individual's  job  training  requirements  and  upgrade  training. 

2. 2. 2. 3  Support  to  the  ULC  by  Other  Squadron/Base  Departments 

ISSUE: 

What  existing  organizational  resources  can  assist  the  ULC  In  reaching 
ATCBT  goals? 

RESULTS: 

During  the  prototype  evaluation,  several  Incidents  occurred  In  which 
support  was  provided  by  other  departments.  These  may  serve  as  a  model  for  the 
future. 
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First,  although  the  ULC  monitors  were  trained  to  perform  preventive 
maintenance.  Including  cleaning  the  Inside  of  computers  and  running  diagnostic 
programs  to  Isolate  faulty  components,  they  were  limited  In  technical  knowledge 
to  remove  and  replace  components.  At  Rhein  Main,  the  systems  section  wh^ch 
cares  for  CAPS,  helped  out  by  Isolating  a  faulty  hard  disk  and  then  swapping  a 
good  disk  from  another  down  machine.  This  assistance  provided  minimal  downtime 
for  one  machine.  The  skills  required  were  not  part  of  those  taught  to  the  ULC 
monitor,  but  are  In  the  repertoire  of  almost  any  computer  technician  familiar 
with  personal  computers. 

The  second  Incident  occurred  at  Charleston  AFB.  The  phone  switching 
system  (the  on-base  PBX)  turned  out  be  an  older,  mechanical  type  which  can  be 
very  noisy  for  the  data  communications  needed  for  ATCBT.  The  squadron 
communications  experts  assisted  In  determining  possible  corrective  actions. 

CONCLUSIONS/RECOMMENOATIONS : 

The  ATCBT  User  Guide  should  Include  discussions  of  possible  squadron  and 
base  resources  to  use  when  available  and  when  approved.  These  resources  may  be 
especially  Important  to  non-CONUS  sites.  HQ  MAC  may  want  to  provide  a  formal 
statement  of  support  required. 

Another  form  of  support  will  be  encountered  during  the  implementation  of 
sites  with  "satellites."  For  example,  Oakland  International  Airport  Is  a 
satellite  to  Travis  AFB.  It  will  be  necessary  to  establish  who  needs  the 
skills  and  Is  responsible  for  what  tasks.  That  Is,  are  the  satellites 
functioning  as  ULCs  or  as  a  remote  site  to  the  central  ULC?  Each  site 
Implementation  should  have  a  designated  Individual (s)  to  accomplish  the  tasks 
described  In  Section  2. 2. 2.1. 

2. 2. 2. 4  Support  to  the  ULC  from  the  MOC  and  the  Site  Installation  Team 

ISSUE: 

How  can  ULC  skills  be  supplemented  by  other  MAC  expertise  available? 

RESULTS: 

During  prototype  Implementation,  the  contractor  visited  each  site  from 
three  to  six  times,  kept  frequent  phone  contact,  and  responded  to  questions  and 
problems  which  occurred.  Many  of  the  contacts  were  required  because  the  system 
was  In  the  prototype  stage  and  they  will  not  occur  during  the  command-wide 
Implementation.  However,  as  each  of  the  sites  are  Implemented,  It  should  be 
expected  that,  regardless  of  Initial  training  and  support  materials  such  as  the 
ATCBT  User  Guide,  the  ULCs  will  have  more  questions  and  unanticipated  problems, 
or  personnel  turnover  requiring  new  support. 

CONCLUSIONS/RECOMMENOATIONS: 

ATCBT  field  support  should  be  transitioned  from  the  contractor  to  the  MOC. 
The  HOC  participated  In  all  the  prototype  installations,  surveyed  the  test 
sites.  Is  on  the  DON  net,  and  has  been  kept  abreast  of  questions  from  the 
field.  ISO,  Inc.  should  continue  training  the  MOC  to  provide  this  operational 
support  function  and  act  as  an  additional,  but  second-line,  resource  to  the  MOC 
and  ULCs. 
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2.2.3  Lesson  Management 


During  coanand-wlde  Implementation*  approximately  200  hours  of  CBT 
Instruction  will  be  developed,  validated,  and  disseminated  to  the  field  by  the 
MDC.  In  addition,  revisions  to  lessons  already  In  the  Inventory  will  be 
required  with  updates  distributed  to  the  field.  Section  2. 3. 4. 3  covers 
specific  MDC  lesson  r.anagement  procedures.  However,  Figure  2-7  presents  the 
elements  for  lesson  manageme'^t  at  the  ULC. 


INPUT 

— >  PROCESS 

— > 

OUTPUT 

New  lessons 

0  Distribution  by  M)C 

0 

Controlled  Inventory 

Lesson  Updates 

to  ULC 

0  Distribution  by  ULC 

up  to  date  at  all 
locations 

to  Work  Station 

0  Dissemination  of  - 
Catalogs  by  MDC 
and  ulC 

0 

Catalog  Information 
complete 

Figure 

2-7.  ATCBT  Lesson  Management  Methods 

ISSUE: 

What  experience  was  gained  during  the  prototype  evaluation  to  suggest 
lesson  control  and  distribution  procedures? 

RESULTS; 

During  the  evaluation.  It  was  necessary  to  send  out  new  disks  to  each  site 
for  three  of  the  nine  lessons.  These  lessons  were  revised  to  Include 
unanticipated  responses.  In  order  to  make  lesson  updating  easy  at  the  ULC.  a 
floppy  disk  was  mailed  with  an  MS-DOS  .BAT  file  which  contained  all  comnands 
necessary  for  updating  the  lessons.  The  ULC  monitors  typed  "UPDATE”  at  each 
station.  ISD,  Inc.  also  distributed  new  CONTROL  boot  disks  which  contained 
changes  (Invisible  to  the  users)  to  lesson  control  files  for  the  purpose  of 
downloading  the  lessons, 

CONCLUSIONS/RECOMMENDATIONS: 

These  specific  procedures  for  the  MDC  and  the  ULC  need  to  be  documented 
for  dissemination  of  the  large  number  of  lessons  scheduled.  ULCs  should  have 
the  option  of  deciding  whether  to  copy  disks  or  download. 

In  addition,  the  CONTROL  software  and  data  management  programs  need  to  be 
modified  to  make  updating  simpler  and  reduce  the  lesson  Information  files. 

Finally,  the  procedures  for  updating  and  disbursing  the  ATCBT  catalog  of 
lessons  to  all  levels  of  users  must  be  established. 
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2.2.4  Equipment  and  Software  Management 


The  ATCBT  terminals  are  Intended  to  be  available  at  work  stations  on  a  24- 
hour,  continuous  operating  basis.  The  work  stations  are  In  passenger 
terminals,  freight  areas,  and  other  areas  remote  from  the  ULCs.  Figure  2-8 
presents  the  elements  for  equipment  and  software  management. 


INPUT  — > 


PROCESS  — >  OUTPUT 


Computer  hardware  and 
software  at  work  sta¬ 
tions  In  24-hour, 
365-day  operations 


0  Operational  tasks  at 
all  stations 
0  Preventive  mainten¬ 
ance 

0  Corrective  mainten¬ 
ance 


0  Maximum  availability 
of  machines 
0  Provide  self- 
sufficiency  at 
squadron  locations 
remote  from  ULC 


Figure  2-8.  ATCBT  Equipment  and  Software  Management  Methods 


ISSUE: 

What  operation  and  maintenance  procedures  need  to  be  followed  by  ULC 
monitors  and  other  personnel? 

RESULTS: 

0  ULC  Monitor  Operations 

Several  Incidents  occurred  which  demonstrated  the  need  to  provide 
more  specific  reference  Instructions  to  the  ULC  on  operations  which 
had  not  been  sufficiently  described.  These  operations  Included  (I) 
backing  up  student  registration  and  lesson  registration  files  on 
floppy  disks,  (2)  backing  up  performance  data  files,  (3)  maintalninq 
a  library  of  disks  of  various  software,  (4)  formatting  disks,  (5) 
using  DON  to  transmit  data,  and  (6)  alternative  data  collection  on 
floppy  disks  when  communications  were  not  operable.  Procedures  to 
explain  the  why  and  how  were  written  as  the  need  arose.  In  most 
cases,  the  write-ups  went  through  at  least  one  revision  after 
validation  by  ULC  testing. 

0  Work  Station  Operations 

The  ULC  monitors  had  no  problems  operating  the  equipment.  However, 
there  were  several  Incidents  at  the  work  stations  which  pointed  to 
the  need  for  others  to  perform  basic  operations  such  as  power  on/off, 
setting  the  systems  for  booting  up,  and  assisting  new  personnel  on 
the  systems  at  the  work  stations. 

When  severe  thunderstorms  occur  at  Charleston  and  Dyess,  the  policy 
Is  to  turn  off  all  computers;  even  with  power  surge  protection.  This 
means  the  systems  must  be  turned  on  later  with  the  proper  disk 
Inserted.  Since  this  may  occur  when  a  ULC  monitor  Is  off  shift,  or 
even  during  the  day  when  other  duties  are  being  performed.  It  means 
someone  else  must  do  It.  Questions  also  arise  at  the  work  stations 


from  new  trainees  about  how  to  use  the  terminals,  the  keyboard,  the 
lessons  available,  etc. 

Preventive  Maintenance  Procedures 

The  need  for  several  new  preventive  maintenance  procedures  was 
demonstrated  during  the  evaluation.  For  example,  at  Charleston  AFB 
an  already  "dirty"  environment  was  made  worse  when  the  forced  air 
heating  was  turned  on.  A  large  amount  of  soot  and  dust  found  Its  way 
Into  the  two  machines  In  the  freight  area,  as  well  as  the  one  at  the 
ULC.  All  three  went  down  and  would  not  respond  normally  over  a 
period  of  mid-December  to  mid-January. 

The  problem  was  solved  by  cleaning  the  Inside  of  the  machines  with  a 
small,  battery-operated  vacuum,  static-free  cloths,  and  a  pencil 
eraser.  The  ULC  monitor  was  shown  how  to  do  this  and  was  provided 
with  written  Instructions.  No  further  problems  occurred.  It  should 
be  noted  that,  at  the  passenger  terminal,  the  section  supervisor  had 
been  cleaning  around  the  outside  of  the  machine  regularly  with  a  hair 
blow  dryer.  This  machine  was  very  clean  Inside  when  checked. 

Other  procedures  which  were  needed  Included  (1)  making  new  boot  disks 
(which  get  dirty  or  are  handled  Incorrectly  In  the  24-hour 
operations),  and  (2)  reformatting  hard  disks  when  they  crash  from 
power  outages  or  have  other  noncritical  hardware  malfunctions.  The 
ULC  monitors  were  able  to  perform  these  tasks  when  given  written 
Instructions  and  the  necessary  resources. 

Corrective  Maintenance  Procedures 

Maintenance  management,  both  preventive  and  corrective,  was 
considered  to  be  one  of  the  most  critical  areas  to  be  resolved  for 
command-wide  Implementation.  It  was  recommended  by  the  contractor 
that  no  commercial  contract  be  made  for  maintenance  of  the  hardware. 

A  90-day  warranty  was  provided  by  AT4T  for  equipment  purchased  from 
them.  During  Installation,  ISD,  Inc.  took  responsibility  fcr 
contacting  AT&T  or  replacing  other  equipment  under  warranty  If  It  did 
not  work  upon  "opening  the  box."  In  addition,  the  warranty  covered 
problems  with  a  display  monitor  and  modem  at  Charleston  and  a 
keyboard  at  the  MDC.  Actions  were  taken  by  personnel  at  the  site 
with  assistance  from  ISD,  Inc. 

Problems  after  the  warranty  period  Included:  (1)  none  at  Charleston 
AFB,  except  for  Internal  cleaning  required  as  preventive  maintenance, 
(2)  a  bad  disk  and  possible  processor/mother  board  defect  at  Rhein 
Main  AB,  (3)  none  at  Dyess  AFB,  and  (4)  a  possible  processor/mother 
board  defect  at  the  MDC.  The  problems  at  both  Rhein  Main  and  the  MDC 
are  pending  corrective  actions.  Form  9's  must  be  filled  out,  vendors 
Identified  for  work,  and  the  mechanics  of  getting  the  machines  or 
components  to  a  repair  vendor  are  In  process. 


CONCLUSIONS/RECOMHENOATIONS: 

The  ATCBT  User  Guide  needs  to  have  operating  instructions  and  preventive 
maintenance  procedures  added  for  both  the  ULC  monitor  and  supervisors  at  work 
stations.  Reference  material  should  be  included  in  the  User  Guide  for 
preventive  and  corrective  maintenance  actions*  including  use  of  the  form  9's, 
how  to  find  vendors*  where  to  get  base  support*  and  how  to  specify  local 
maintenance  contract  needs.  An  oversight  in  the  prototype  evaluation  was  that 
“ownership"  of  the  equipment  was  not  officially  transferred  to  the  squadrons. 
This  meant  they  could  not  get  approval  for  repairs.  This  has  been  corrected 
and  will  be  done  in  all  future  implementations. 

The  Air  Force-wide  small  computer  maintenance  contract*  to  be  awarded  in 
the  next  several  months*  should  be  studied  to  determine  its  usefulness  to 
ATCBT. 

2.2.5  Facilities  Management 

As  shown  in  Figure  2-9*  facilities  Includes  power*  communications*  space* 
furniture*  supporting  material*  and  environmental  factors. 


INPUT  — > 


PROCESS  — >  OUTPUT 


0  Power  requirements  o  Prepare  facility 
0  Communication  re-  o  Maintain  facility 
quirements 
0  Space/furniture/ 
support  material 
requirements 
0  Environmental 
factors 


0  Minimize  equipment 
downtime 

0  Provide  adequate 
area  for  work 
station  learning 


Figure  2-9.  ATCBT  Facility  Management  Methods 


ISSUE: 

What  are  the  facility  preparation  and  maintenance  requirements  for  ATCBT? 

RESULTS: 

0  Power 

Power  outlets  sufficient  for  each  machine  were  required.  Normally  in 
CONUS*  this  meant  one  outlet  (receptacle)  per  machine  since  each 
machine  was  connected  through  a  surge  protector.  If  an  external 
modem  or  printer  was  at  the  station*  these  were  also  connected 
through  the  surge  protectors.  The  surge  protectors  proved  necessary 
and  adequate.  Electrical  storms  at  Charleston  did  cause  problems* 
but  the  surge  protectors  avoided  potentially  serious  problems  with 
the  computers. 


I 
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Two  other  factors  are  relevant  to  European  and  Aslan  installations 
where  the  power  will  not  be  llOV,  60  Hz.  Experience  at  Rhein  Main 
(220V,  50  Hz)  illustrates  the  alternatives.  The  AT&T  6300  has  a 
power  supply  which  is  set  for  the  U.S.  IIOV,  60  Hz,  but  can  be 
switched  to  the  European  220V,  50  Hz  setting  via  jumpers  in  the  back 
of  the  chassis.  However,  at  Rhein  Main,  transformers  were  used  since 
(1)  the  printers  and  external  modems  were  not  switchable  to  other 
than  llOV,  and  (2)  transformers  were  readily  available. 

Two  problems  occurred.  First,  surge  protectors  could  not  be  used 
unless  transformers  of  higher  ratings  than  usual  were  used.  The 
surge  protectors  pulled  so  much  power  that  the  entire  ground  floor  of 
the  ULC  building  was  blacked  out  when  even  a  2000  watt  transformer 
was  used  and  a  fuse  was  blown.  Second,  an  older,  faulty  transformer 
was  used  in  error  one  day  at  the  ULC  which  resulted  in  another  blown 
fuse  and  a  "crash"  of  the  hard  disk.  The  "crash"  resulted  in  loss  of 
data  and  data  management  programs. 

Communications 

Communication  orders  for  the  prototype  implementation  called  for  a 
Class  C  line  with  an  RJ-11  jack  at  each  CBT  terminal  for  on-base  data 
and  lesson  transmission.  In  addition,  a  commercial  line  at  the  ULC 
was  requested  for  access  to  DON  and  the  uploading  of  data  summaries 
to  the  MDC.  Variations  In  the  actual  implementation  ocurred  as 
follows. 

First,  the  1st  MAPS  at  Dyess  AFB  needed  their  two  machines  in  the 
same  room.  They  will  move  to  another  building  in  several  months  at 
which  time  the  machines  will  be  about  30  feet  apart.  They  are 
currently  2  to  3  feet  apart.  It  did  not  make  sense  to  install  Class 
C  lines.  Instead,  the  ATCBT  CONTROL  software  was  extended  to  support 
a  null  modem,  direct  cabling  of  two  machines  with  no  modems.  A 
commercial  line  was  installed  for  DON. 

Second,  Rhein  Main  AB  could  obtain  either  Class  C  or  dedicated  lines. 
Class  C  lines  were  chosen  because  dedicated  lines  were  more  expensive 
and  not  as  easily  moved.  No  problems  were  experienced  with  the  Class 
C  lines.  It  should  be  noted  that  the  base  switching  system  is 
overloaded,  which  results  in  daytime  periods  with  busy  signals  all 
over  the  base.  The  ATCBT  system  was  not  hindered  by  such 
circumstances. 

The  Rhein  Main  AB  site  was  a  good  test  of  the  ATCBT  communication 
concept  because  of  the  differences  from  CONUS.  Special  requirements 
for  modems  were  incorporated  in  the  final  specifications  due  to  the 
European  phone  system  and  CCITT  protocol.  The  ATCBT  system  was  not 
hindered  by  such  circumstances.  The  ULC  modem  hookup  to  DON  is  not 
possible  in  Europe.  However,  the  ULC  uses  another  squadron  DDN 
facility  with  a  direct  line  to  a  DDN  host. 

Third,  Charleston  AFB  installed  Class  C  lines  and  a  commercial  line 
for  DDN.  After  several  attempts  and  tests  of  the  Class  C  lines,  it 
was  concluded  that  the  noise  on  the  lines  was  just  too  much.  The 
ATCBT  messages  and  data  were  being  distorted  or  destroyed  by  the  line 


noise  such  that  about  40  to  60t  of  the  calls  could  not  be  completed 
successfully.  Based  on  this  experience,  a  set  of  minimum 
specifications  have  been  developed  for  communications  orders.  The 
Charleston  base  switching  system  is  the  older,  mechanical  type  for 
which  not  much,  if  any,  line  conditioning  could  be  done.  The  plan  is 
to  implement  dedicated  lines. 

It  should  be  noted  that  the  Charleston  base  switching  system 
represents  the  worst  possible  case  and  is  unlikely  to  be  found  at 
other  bases.  In  fact,  most  bases  are  being  upgraded  to  all  digital 
switching,  the  latest  in  the  state-of-the-art.  Dyess  AFB  already  has 
this  system.  The  new  systems  are  made  for  integrated  voice  and  data 
networks  and  will  support  the  ATCBT  operational  concept  well.  In 
addition,  it  may  be  possible  to  increase  the  rates  of  transmissions 
thereby  making  the  data  transmission  even  less  visible  than  currently 
and  lesson  downloads  more  feasible  (they  take  10  to  20  minutes  at  a 
1200  baud  rate). 

Space/Furni ture/Materi al  Requi rements 

None  of  the  three  prototype  sites  used  any  special  furniture.  Tables 
and  standard  desks  and  chairs  are  being  used  satisfactorily. 
However,  all  bases  indicated  intentions  to  put  personal  computer  type 
carrels  in  the  work  stations.  Because  all  sites  had  nylon  carpeting 
and  some  had  severe  electrical  storms,  ISO,  Inc.  recomnended  that 
they  acquire  static-reducing  floor  mats.  Most  stations  have  these. 
The  space  required  for  any  work  station  is  no  more  than  it  takes  for 
a  small  work  table  or  desk.  In  several  instances,  the  CBT  terminals 
are  placed  in  unused  corners  of  rooms  where  they  are  accessible  but 
not  in  the  main  flow  of  traffic.  In  other  cases,  the  stations  are  in 
small  rooms  with  a  door  which  can  be  closed  if  the  area  is  noisy. 
All  sites  were  g-iven  starter  cleaning  kits,  including  disk  head 
cleaning,  CRT  cleaning,  ate.,  to  maintain  the  machines.  These  kits 
cost  a  minimal  amount  zud  are  easily  replaced.  Other  consumables 
include  printer  paper  and  ribbons,  floppy  disks,  and  disk  storage 
containers. 

Environmentol  Factors 

Environmental  factors  of  relevance  include  heat,  cold,  humidity, 
electricity/stutic,  dirt  in  the  air,  and  items  such  as  cigarette 
smoke,  drinks,  and  food.  All  prototype  sites  put  signs  at  the  work 
stations  which  prohibit  any  food,  drink,  or  smoking  at  the  terminals. 
Beyond  that,  other  precautions  already  mentioned  were  surge 
protectors  (except  Rhein  Main),  static-protect  floor  mats,  and 
preventive  maintenance  cleaning.  These  precautions  proved  adequate. 
No  problems  have  been  caused  by  environmental  factors  which  cannot  be 
prevented  by  these  precautions.  The  terminals,  in  most  instances, 
were  operating  24  hours  a  day  in  a  relatively  hostile,  non-office 
type  environment. 


CONCLUSIONS/ RECOHMENDAT IONS: 


Transformers  should  be  used  In  non-CONUS  sites  since  not  all  equipment 
will  be  convertable.  The  contractor  should  investigate  the  use  of  higher-rated 
transformers  for  non-CONUS  sites  so  that  surge  protectors  may  be  used  with 
transformers.  New  transformers  should  be  acquired  and  they  should  be  planned 
for  replacement  periodically  (probably  1  to  2  years)  since  they  do  deteriorate 
and  this  may  damage  ATCBT  equipment. 

A  summary  of  all  sites  needs  to  be  completed  to  determine  the  capability 
to  meet  the  minimum  communication  specifications  and  to  determine  the  type  of 
PBX  system  available  so  that  the  ATCBT  CONTROL  programs*  modems*  and  port 
switching  can  be  configured  properly.  Orders  must  be  issued  at  least  90  days 
in  advance.  CONTROL  software  should  be  modified  to  provide  for  cases  where 
dedicated  lines  and  digital*  integrated  systems  are  used  for  communications. 
Error  protocols  and  retry  algorithms  should  be  enhanced. 

The  ATCBT  operational  concept  of  (1)  placing  the  terminals  at  the  work 
stations*  and  (2)  requiring  around-the-clock  operation  in  a  non-protected 
environment  seems  sound.  The  stations  are  operating  normally.  The  number  of 
problems  which  have  occurred  do  not  exceed  those  expected  in  a  more  protected* 
8-hour-a-day  environment. 

2.3  Production  Component 

The  ATCBT  production  configuration  was  designed  to  meet  the  needs  of  the 
HOC  as  a  centralized  CBT  production  capability.  To  better  understand  these 
needs*  a  brief  description  of  the  responsibilities  of  the  MDC  is  provided 
below. 

The  MDC  is  the  Office  of  Primary  Responsibility  (OPR)  for  a  number  of 
training  programs  (including  both  slide/tape  and  programmed  text).  It  will  be 
the  OPR  for  the  ATCBT  lessons  as  they  are  developed  during  command-wi de 
implementation.  As  an  OPR*  It  is  tasked  with  creating*  reviewing, 
standardizing*  distributing*  and  purging  training  materials  for  the  ULCs 
worldwide.  The  MDC  also  serves  as  a  preview/review  agency  for  all  HQ  AFRES  Air 
Transportation  training  materials. 

During  the  evaluation  period*  the  MDC  staff  was  tasked  with  inventorying 
all  slide/tape  programs  (regardless  of  OPR)  and  indicating  their  status  as 
current*  requiring  revision*  or  obsolete.  As  a  result*  the  Transportation 
Training  Program  Catalog  was  developed  by  HQ  AFRES*  with  the  assistance  of  the 
MDC*  listing  all  available  training  material  for  AFSC  605XX*  including  printed 
materials*  slide/tape  programs*  and  CBT  lessons  (as  they  become  available). 
The  MDC  coordinates  with  HQ  AFRES  to  update  this  catalog  of  ULC/AFRES  training 
materials  on  a  quarterly  basis. 

The  evaluation  of  the  production  component  at  the  MDC  focused  on  four 
specific  objectives: 


1.  The  ease  of  use  of  the  ATCBT  authoring  environment. 

2.  The  efficiency  of  the  graphics  development  process. 

3.  The  adequacy  of  the  prototype  system  configuration. 

4.  The  requirements  for  lesson  production  and  maintenance. 

The  results  of  the  evaluation  are  organized  Into  four  categories  In  the 
fol lowing  subsections:  (1)  the  authoring  environment.  (2)  graphics 

development,  (3)  the  prototype  configuration,  and  (4)  lesson  production, 
maintenance  and  management. 

2.3.1  Authoring  Environment 

There  are  several  Issues  to  consider  In  evaluating  the  ease  of  use  of  the 
authoring  environment.  These  Issues  center  on  the  ability  of  non-programners 
to  author  lessons,  the  usefulness  of  the  authoring  training  workshops  and  User 
Guide,  the  "learning  curve"  and  time  to  develop  lessons,  and  the  requirement 
for  modifications  and  enhancement  to  the  ATCBT  authoring  prototype.  The 
Issues,  results,  and  conclusions/ recommendations  are  described  In  detail  below. 

2.3.1. 1  Non-Programmers  as  Authors 

ISSUE: 

The  major  criterion  for  designing  the  production  component  was  the 
requirement  for  non-programmers  to  author  CBT  lessons.  The  Issue  becomes,  "Can 
the  MDC  staff  produce  CBT  lessons  using  the  ATCBT  authoring  environment?" 

RESULTS: 

The  MOC  staffing  level  Is  set  at  six  enlisted  personnel  with  Air 
Transportation  Instructor  experience.  There  were  three  N)C  personnel  at  the 
beginning  of  the  evaluation  In  June  with  the  fourth  added  In  August,  the  fifth 
In  November,  and  the  sixth  In  December.  Therefore,  not  all  members  of  the 
staff  have  extensive  hands-on  authoring  experience  at  this  point.  One  CBT 
lesson  was  completed  during  the  evaluation  and  three  CBT  lessons  are  In 
progress  and  scheduled  for  completion  by  1  April  1986. 

CONCLUSIONS/RECOMMENDATIONS: 

The  design  of  the  ATCBT  authoring  environment  does  provide  the  non¬ 
programmer  staff  of  the  MDC  with  the  capability  to  author  CBT  lessons.  The  MOC 
personnel  were  able  to  research  a  topic  area,  design  a  CBT  lesson  using  the 
lesson  specification  guidelines.  Input  the  lesson  content  unit  by  unit,  "run" 
the  lesson  as  a  student  would  see  It,  and  then  edit  the  units  for  better 
presentation.  The  CBT  lesson  completed  during  the  evaluation  period  Is  well 
constructed,  presented  In  an  Interesting  manner  and  Interactive  for  a  student. 
The  authoring/editing  user  guide  and  hands-on  training  workshops  provided  by 
ISO,  Inc.  appear  to  be  useful  In  preparing  the  MOC  personnel  to  author  CBT 
lessons. 
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2.3. 1.2  Slide/Tape  Versus  CBT  Production 

ISSUE: 

As  an  OPR*  the  HOC  Is  tasked  to  produce  slide/tape  programs  for  the  ULCs. 
It  Is  anticipated  that  this  requirement  will  eventually  be  replaced  by  CBT 
lesson  production  and  maintenance.  However,  during  the  evaluation  period,  the 
MOC  staff  developed  five  slide/tape  programs,  starting  with  research  through 
editing  to  final  production.  Since  the  topic  of  one  of  the  slide/tape  programs 
was  also  designated  as  a  CBT  lesson,  this  provided  an  excellent  opportunity  to 
compare  slide/tape  production  time  against  CBT  production  time.  The  question 
Is,  "Is  Is  possible  that  CBT  development  can  reduce  the  time  required  to 
produce  training  for  the  ULCs?" 

RESULTS: 

Development  of  a  slide/tape  program  on  the  topic  "Terminal  Security/Anti- 
Hijacking  Procedures"  began  3  September  1985.  Development  of  a  CBT  lesson  on 
the  same  topic  began  23  December  1985.  The  CBT  lesson  was  completed  by  14 
February  1986,  while  the  slide/tape  program  Is  still  In  process.  Most  of  the 
time  delay  associated  with  the  slide/tape  production  Is  the  result  of 
audiovisual  coordination  with  DAVA.  OAVA  Is  responsible  for  all  slide 
reproduction,  audio  production,  and  mailing  to  the  55  stations  supported  by 
MAC.  There  Is  a  significant  amount  of  stagnation  time  before  slide/tape 
programs  are  disseminated  to  the  field  for  training.  This  time,  plus  the 
additional  costs  of  reproductions  and  narrations  Incurred  by  DAVA,  need  to  be 
considered  when  looking  at  overall  slide  tape  production. 

The  times  devoted  to  the  major  production  stages  for  each  medium  are  shown 
In  Table  2-4.  The  times  are  based  on  production  time  sheets  kept  by  the  MDC 
author.  To  date,  the  total  number  of  hours  for  the  CBT  lesson  production  was 
slightly  more  than  that  for  the  slide/tape  production.  However,  the  slide/tape 
production  has  not  been  finished  and  actual  hours  for  DAVA  processing  are  not 
Included. 


Table  2-4 

Hours  for  MDC  Lesson  Production  In  Two  Media 


Typing/ 

Review/ 

Graphics/ 

Media 

Research 

Development 

Input 

Editing 

Photography 

Total 

CBT 

60.00 

82.00 

45.00 

41.00 

82.00 

310.00 

S/1 

24.50 

74.00 

32.25 

80.50 

13.00 

224.25 

J 
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CONCLUSIONS/RECOMMENOATIONS : 

The  development  of  CBT  lessons  can  decrease  the  time  required  to  produce 
training  for  the  ULCs.  Even  during  the  Initial  “learning  curve"  for  CBT 
authoring,  the  number  of  hours  per  hour  of  Instruction  Is  comparable  to  those 
required  for  cllde/tape  development.  In  addition,  since  all  of  the  CBT 

production  occurs  In-house  at  the  MDC,  coordination  time  with  OAVA  Is 
eliminated,  and  the  total  time  to  get  training  out  to  the  ULCs  Is  reduced. 

2. 3. 1.3  Modifications  to  Authoring  Prototype 

ISSUE: 

The  ATCBT  authoring  prototype  produces  standardized  lessons  containing  a 
set  of  structured  Instructional  units.  The  author  only  needs  to  respond  to  a 
sequence  of  prearranged  prompts  In  order  to  generate  a  lesson.  The 

Instructional  units  and  prompts  In  the  ATCBT  prototype  were  designed  by  ISO, 

Inc.  with  Inputs  from  the  HOC  staff.  The  Issue  here  Is,  “Does  the  ATCBT 

authoring  prototype,  as  currently  oeslgned,  meet  the  needs  of  the  MDC  operating 
In  a  CBT  lesson  production  mode?" 

RESULTS: 

Based  on  MDC  comments,  the  ATCBT  prototype  needs  to  be  modified  to  better 
meet  the  authoring  requirements  of  the  MDC.  Specific  problems  and  solutions 
are  provided  below. 

0  Introduction 

Problem:  Currently  the  prototype  Is  only  set  to  Introduce  a  lesson 
with  a  text  title  page.  At  times,  a  graphic  title  page  might  be  more 
effective  as  an  Introduction. 

Solution:  An  option  should  be  available  to  the  author  to  allow 

either  a  text  or  graphic  title  page. 

0  Learning  Objectives 

Problem:  The  learning  objectives,  as  currently  stated  In  the 
prototype  lessons,  are  not  consistent  with  the  "Gronlund“-type  format 
used  by  the  Air  Transportation  Training  Command  (ATTC)  formal  school. 

Solution:  Revise  the  authoring  prototype  to  specifically  prompt  an 

author  to  Input  learning  objectives  In  the  format  used  by  the  formal 
school . 

0  References 

Problem:  The  references  used  to  develop  a  lesson  are  not  formally 

displayed  In  the  actual  lesson  run-time,  possibly  causing  future 
problems  In  Identifying  lesson(s)  that  are  subject  to  update  due  to 
poHcy/technlcal  manual  change. 


2-36 


Solution:  Revise  the  authoring  system  to  prompt  an  author  to  Input 

the  references  at  the  lesson  and  segment  levels  to  provide  (1)  a 
revision  audit  trail  and  (2)  more  authority  to  the  content. 

Lesson  Title 

Problem:  There  are  currently  two  locations  In  which  an  author  must 

Input  the  lesson  title,  causing  duplicate  effort. 

Solution:  Change  the  prototype  to  either  have  only  one  occurrence  of 
the  title  In  the  lesson,  or  store  the  title  and  redisplay  It  as 
necessary. 

Text  Entry 

Problem:  Currently,  text  entry  Is  done  line  by  line  with  no  option 

to  return  to  previously  entered  lines.  This  Is  especially  a  problem 
when  the  main  body  of  a  segment  must  be  entered  and  structured  to 
provide  "full-page"  text  screens. 

Solution:  Revise  the  prototype  to  provide  on-screen  editing  of  major 
portions  of  text. 

Color  Standardization 

Problem:  Currently,  prompts  for  student  entry  are  presented  In 
yellow.  In  order  to  further  establish  conventions  for  standardized 
lesson  formats,  a  common  color-coding  of  text  must  be  established. 

Solution:  Utilize  the  following  color  conventions  for  future 

lessons:  default  color  of  green  for  general  text;  magenta  and  white 

for  special  emphasis  of  a  word  or  words;  turquoise  for  notes,  yellow 
for  cautions,  and  red  for  warnings,  hazards,  dangers  and  safety 
precautions. 

Prompts  to  Press  a  Key 

Problem:  When  the  color  monitor  Is  In  graphics  mode,  the  prompt  to 

"Press  any  key  to  continue"  occurs  at  the  bottom  of  the  screen  with 
the  lower  half  of  the  letters  cut  off. 

Solution:  The  developers  of  CDS  II  should  be  able  to  redisplay  the 

prompt  up  one  line  so  It  Is  no  longer  "clipped". 

Entry  of  Lesson  Filename  by  Author 

Problem:  There  Is  no  Information  at  startup  for  the  author  to  enter 

a  lesson  filename  no  longer  than  8  characters. 

Solution:  The  developer  of  COS  II  should  explicitly  Include  a  prompt 
for  a  filename  of  no  more  than  eight  characters. 


CONCLUSIONS/RECOMMENOATIONS : 


The  modifications  to  the  ATCBT  authoring  prototype  should  be  one  of  the 
Initial  tasks  performed  under  command-wide  Implementation.  Lessons  using  the 
current  prototype  will  need  to  be  reviewed  and  revised  to  reflect  the  changes 
and  ensure  the  format  Is  standardized.  The  more  CBT  lessons  In  the  Inventory 
that  are  produced  prior  to  prototype  modification,  the  greater  the  lesson 
revision  requirement. 

2. 3. 1.4  Enhancements  to  the  Authoring  Prototype 

ISSUE: 

The  current  ATCBT  authoring  prototype  was  designed  to  provide  a  basic 
structure  and  standardized  format  for  lessons.  During  the  development  of 
the  nine  CBT  lessons  for  prototype  evaluation.  It  became  clear  that  there  was  a 
requirement  to  enhance  the  authoring  prototype. 

RESULTS: 

An  upgrade  to  the  CDS  II  authoring  software  Is  now  available  to  enhance 
the  current  ATCBT  prototype  as  well  develop  new  prototypes.  New  prototypes 
are  based  on  specific  content  requirements  that  extend  across  several  lessons. 
There  appear  to  be  a  number  of  content  area/presentation  requirements  that 
could  warrant  specific  prototypes.  For  example,  filling  In  PACS  screen  masks 
and  simulating  ADAM  III  terminal  displays  are  both  content  areas  In  which 
specific  authoring  prototypes  could  be  made  available. 

CONCLUSIONS/RECOMMENDATIONS : 

It  Is  recommended  that  the  current  version  of  the  CDS  II  authoring 
software  be  upgraded.  The  latest  version  of  CDS  II  Includes  a  number  of  new 
features  that  were  not  available  when  developing  the  original  authoring 
prototype.  However,  the  update  will  not  Impact  the  lessons  previously 
developed.  The  requirement  for  enhancing  the  current  prototype  and  developing 
additional  prototypes  to  support  MDC  lesson  production  should  also  be  Included 
as  one  of  the  Initial  tasks  for  the  command-wide  Implementation, 
authoring/editing  user  guide  will  also  be  revised  to  reflect  the 
changes/upgrade  In  the  authoring  prototype. 

2. 3. 1.5  Software  for  Field  Replication 

ISSUE: 

The  M)C  does  not  have  the  software  to  replicate  the  workstation  training 
environment. 

RESULT:  To  replicate  the  training  environment,  the  MDC  needs  to  license 

copies  of  the  CONTROL  software  and  CDSX,  the  lesson  run-time  software  for 
lessons  developed  under  CDS  II.  Without  CDSX  and  CONTROL  In  combination, 
lessons  cannot  be  presented  as  they  are  to  students  In  the  field. 
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CONCLUSIONS/RECOHMENDATIONS : 


The  MDC  production  stations  should  be  upgraded  for  connand-wide 
Implementation.  Copies  of  both  the  CONTROL  software  and  CDSX  should  be 
acquired. 

2.3.2  Graphics  Development 

The  MDC  used  a  mouse  and  standard  graphics  software  package  to  develop 
picture  files  (PICS)  for  the  CBT  lessons.  These  files  are  accessible  by  CDS  II 
to  display  full-screen  pictures  within  a  lesson.  During  the  evaluation  phase  a 
library  of  over  150  graphics  was  developed,  based  on  Illustrations  In  technical 
manuals,  CDCs,  and  slides  from  existing  slide/tape  programs.  The  library  of 
graphics  was  developed  to  provide  a  resource  for  later  anticipated  CBT 
programs. 

The  creation  of  this  library  of  picture  files  provided  a  good  baseline  for 
evaluating  the  use  of  the  standard  off-the-shelf  graphics  software  package  used 
with  a  mouse.  Specifically,  three  Issues  were  evaluated:  (1)  use  of  an  off- 
the-shelf  graphics  package  by  non-artists  to  create  appropriate  pictures  for 
CBT  lessons,  (2)  alternatives  to  Improve  production  time  for  a  "typical" 
picture,  and  (3)  text  size  when  the  monUor  Is  In  graphics  mode. 

2. 3. 2.1  Use  of  a  Graphics  Package 

ISSUE: 

Can  non-artists  utilize  a  graphics  package  to  develop  pictures? 

RESULTS: 

It  was  determined  that  non-artists  can  Indeed  develop  graphics  with  a 
degree  of  sophistication  that  Is  appropriate  for  the  lesson  content.  Of  the 
150  pictures  created  by  the  MDC  staff,  all  were  of  a  quality  that  reflected  the 
Important,  essential  points  Illustrated  In  the  original  visual  (I.e.,  figures 
from  a  technical  manual  or  35mm  slides  from  slide/tape  programs).  Examination 
of  the  dates  on  the  pictures  files  did  not  show  any  significant  "learning 
curve"  associated  with  the  prototype  Input  device  and  graphics  software.  This 
level  of  production  Is  shown  In  Table  2-5. 

CONCLUSIONS/RECOMMENDATIONS: 

During  the  evaluation  period,  the  MDC  personnel  were  using  the  mouse  as  an 
Input  device  to  develop  graphics  with  a  degree  of  sophistication  that  was 
easily  attainable  and  not  subject  to  Improvement  (because  of  the  restrictions 
of  the  hardware/software).  Thus,  full-screen  graphics  could  be  produced  by 
non-artists  with  little  "learning  curve"  time  expended. 
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2. 3. 2. 2  Production  Time 


ISSUE: 

The  current  mode  of  using  a  mouse  to  produce  graphics  for  inclusion  in 
lessons  has  been  reported  as  being  time-consuming  and  inadequate.  The  issue 
becomes,  "What  alternatives  are  available  to  improve  graphics  production  time?" 

RESULTS: 

The  input  device  supplied  with  the  PC  Paint  software  is  a  Mouse  Systems 
optical  mouse.  The  optical  mouse  was  Initially  chosen  because  it  was  not 
subject  to  wear  and  had  a  projected  long  "lifetime”.  Use  of  the  mouse  to 
input  a  graphic  was  reported  to  be  time-consuming.  Of  the  four  main  steps  in 

graphics  production  (research,  layout,  initial  Tine  drawing  and  color/detail), 

the  initial  line  drawing  using  the  mouse  was  reported  to  take  the  majority  of 
time,  with  later  addition  of  color  and  detail  taking  much  less  time.  Because 
the  initial  drawing  took  the  most  time,  alternate  input/tracing  devices  may  be 
required  in  order  to  speed  production. 

Table  2-5  shows  that  the  type  of  graphic  affected  the  total  production 
time.  For  example,  an  average  of  1.2  hours  was  devoted  to  a  graphic  of  a  form, 
as  opposed  to  almost  four  hours  devoted  to  a  graphic  of  a  piece  of  equipment  or 
other  situational  scene.  Alternate  graphic  devices,  such  as  digitizer  tablets 
and  video  digitizers,  will  Impact  equipment/scene  graphics  the  most,  with 
little  impact  on  illustrations  of  forms.  The  features  of  digitizer  tablets  and 
video  digitizers  are  discussed  below. 

0  Digitizer  Tablets.  There  are  a  variety  of  digitizer  tablets  which 

could  be  interfaced  to  the  ATCBT  system.  These  range  in  price  from 

around  $300  to  $850.  The  current  graphics  development  software  (PC 
Paint)  does  not  directly  support  a  bit  pad,  but  the  Dr.  Halo  graphics 
package  now  available  as  a  COS  II  option  does  have  digitizer  tablet 
Interfaces.  Table  2-6  lists  several  digitizer  tablets  that  interface 
with  the  Dr.  Halo  software  available  for  COS  II  and  available  for 

microcomputers. 
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MUC  Developed  Graphics 


Table  2-6 


Digitizer  Tablets 


Company 

Product 

Resolution 

Size 

Price 

Versa  Computing 

VersaHriter 

320  X  200 

8-1/2“  X  12" 

$299 

GTCO  Corp. 

Micro  Digi-Pad 

200  Ipi 

6"  X  6" 

12"  X  12" 

$400 

NA 

Hitachi 

Tiger  Tablet 

1000  Ipi 

11"  X  11" 

15"  X  15" 

$858 

$1721 

Summagraphics 

MM  Series 

100-1000  Ipi 

6"  X  9" 

11.7"  X  11.7" 

$600 

$800 

Bit  Pad  2 

1-1016  Ipi 

11"  X  11" 

$840 

0  Video  Digitizers.  Video  digitizers  allow  the  capture  and 
digitization  of  photographic  or  videotape  images.  The  systems 
consist  of  an  add-on  board  and  video  camera  along  with  various  image 
processing  software  capabilities.  These  provide  a  digitized  video 
frame  to  whatever  graphics  display  board  is  used  by  the  system.  The 
major  tradeoffs  are  resolution  and  number  of  colors.  In  order  to 
present  a  highly  realistic  color  image*  a  large  number  of  colors  are 
required  (e.g.*  65,000).  A  combination  of  image  thresholding  and 
pseudo-color  capabilities  would  allow  a  photo  image  to  be 
"posterized"  (i.e.,  reduced  to  a  small  number  of  intensity  levels  and 
colors  (therefore  reducing  memory  storage  requirements).  Most  PC 
digitizers  sample  at  between  4  and  8  bits.  That  means  they  produce 
between  16  and  256  grey  level  intensities.  The  number  of  colors  is 
determined  by  the  capabilities  of  the  display  board  and  imaging 
software's  pseudo-color  capabilities. 

Digitized  video  takes  up  greater  amounts  of  memory  storage  than 
simpler  graphics  approaches,  especially  as  the  level  of  resolution 
and  numbers  of  displayed  colors  increases.  Image  compression 
software  decreases  the  memory  requirements  to  a  level  which  would  be 
practical  in  the  current  memory-limited  ATCBT  system.  Use  of  CD- 
ROMs,  videodiscs,  and  optical  recorders  might  make  the  use  of 
realistic  video  images  practical,  but  will  cause  an  increase  in  cost 
of  $1,000  to  $1,500  at  each  delivery  system. 
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Several  video  digitizer  systems  are  listed  In  Table  2-7.  Host 
companies  provide  a  range  of  hardware  and  software  for  Image  capture 
and  digitization,  with  prices  ranging  from  around  $2,000  to  $6,000. 
One  of  these  systems  (Chorus  Data  System's  PC  Eye)  does  not  require 
any  specialized  display  board  for  the  digitized  Image  at  the  delivery 
system.  PC  Eye  produces  PIC  file  formats  that  can  be  accessed  by  Dr. 
Halo  graphics  software.  COS  II  already  supports  Dr.  Halo  graphics, 
so  there  Is  no  additional  cost  to  Implement  special  graphics  routines 
In  COS  II.  If  new  CDS  II  driver  software  Is  required  to  support  any 
new  PIC  file  formats  resulting  from  alternate  video  digitizers,  this 
would  require  a  one-time  cost  of  approximately  $5,000. 

Table  2-7 
Video  Digitizers 


Company 

Product 

Digitize 

Level 

Resolution 

Price 

Imaging  Technology 

PC  Vision  Frame  Grabber 

4-b1t 

8-b1t 

512x480 

$3095 

$3495 

AT&T 

Image  Capture  Board 

TARGA  Frame  Grabber  16 
TARGA  RGB  Camera 

True  Image  Paint  S/W 

256x256 

512x512 

$1295 

$2995 

$1295 

$695- 

1250 

System  Total  (w/o  stand) 

$3280- 

5540 

Chorus  Data  Systems 

PC  Eye  Digitizer 

Camera 

Stand/Tripod,  etc. 

Dr.  Halo  Software 

6-b1t 

8-b1t 

320x200 

640x400 

$610 

$1495 

$270- 

1915 

$215- 

295 

$99 

System  Total 

$1194- 

5849 


CONCLUSIONS/RECOHMENDATIONS: 


An  upgrade  of  Input  devices  to  the  digitizer  tablet  would  better  meet  the 
MDC's  graphics  production  reoulrements.  A  Summagraphlcs  MM  series  tablets  Is 
reconmended  because  of  size  (neither  too  big  or  small),  cost.  Input  resolution, 
and  Its  representation  of  newer  technology.  Its  ability  to  be  used  with  Dr. 
Halo  graphics  software  (which  allows  the  digitized  pictures  to  be  accessed  by 
the  COS  II  lesson  software)  without  the  need  for  additional  graphics  boards  Is 
another  benefit.  Furthermore,  transition  to  Or.  Halo  software  from  the  PC 
Paint  software  will  not  Impact  previously  developed  graphics;  the  PC  Paint¬ 
generated  graphics  can  be  converted  to  a  Or.  Halo  format  with  minimal  time  and 
effort.  The  upgrade  to  a  video  digitizer  would  normally  not  be  advisable  at 
this  time.  Many  digitizers  require  display  boards  that  would  need  to  be 
Incorporated  In  all  delivery  systems.  However,  the  Chorus  Data  System's  PC  Eye 
Interfaces  with  the  Dr.  Halo  software  described  above,  and  does  not  require  a 
special  graphics  boaid  for  delivery.  For  additional  graphics  production,  the 
PC  Eye  system  could  be  acquired  on  a  trial  basis  to.  be  Implemented  at  the  MOC. 

2. 3. 2. 3  Text  Display 

ISSUE: 

The  font  size  used  to  display  text  on  a  picture  file  Is  too  large, 
especially  when  a  form  needs  to  be  Illustrated. 

RESULTS; 

Currently,  when  a  lesson  Is  produced,  the  text  line  to  be  Inserted  on  a 
picture  Is  restricted  to  40  characters  (which  Is  the  state  the  monitor  switches 
to  when  In  graphics  mode).  The  vertical  height  of  characters  Is  also  enlarged. 
This  expansion  of  characters  restricts  the  amount  of  Information  that  can  be 
presented. 

There  are  two  ways  In  which  text  Is  displayed  with  a  picture:  (1)  by 
putting  the  text  onto  the  picture  Itself,  or  (2)  by  creating  a  "window"  In 
which  text  Input  through  the  authoring  system  Is  displayed.  Additional  fonts 
(In  sets  of  five)  can  be  purchased  for  the  current  graphics  software  to  put 
text  onto  the  picture.  Some  of  these  font  sets  contain  smaller  type  faces. 
For  text  entered  through  the  authoring  system,  only  one  single  text  size,  which 
Is  enlarged  when  the  monitor  Is  put  Into  graphics  mode.  Is  currently  available. 
One  of  the  updates  for  CDS  II  Is  to  provide  alternate-size  characters  ("shape 
sets")  In  which  a  smaller  type  face  can  be  accessed  for  text  entry  as 
necessary. 

All  text  fonts  are  limited  to  the  resolution  of  the  screen  (I.e.,  there  Is 
a  minimum  number  of  pixels  required  to  make  one  text  character  distinct  from 
another).  7!ie  standard  AT4T  color  monitor  displays  color  graphics  at  medium 
resolution  (320x200  pixels),  with  higher  resolution  (640x400  pixels)  available 
for  monochrome  characters.  For  graphics  that  require  a  lot  of  text  (e.g.,  AF 
Forms),  an  option  to  switch  to  monochrome  as  needed  may  be  desirable.  The 
possibllllty  of  Implementing  such  an  option  needs  to  be  further  discussed  with 
the  CDS  II  programmers.  The  new  release  of  an  advanced  CDS  II  allows  for  calls 
to  programs  external  to  CDS.  This  may  be  a  way  to  reach  the  monochrome 
resolution. 
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Another  option  (though  costlier)  would  be  to  update  the  color  graphics 
resolution  at  each  production  station  and,  consequently,  each  delivery  station. 
At  the  beginning  of  the  ATCBT  project,  the  standard  graphics  resolution 
available  for  IBM  compatibles  was  320x200  pixels.  Since  then,  a  number  of  new 
high-resolution  boards  with  a  640x400  resolution  have  become  available.  These 
boards  offer  four  times  the  resolution  of  the  current  ATCBT  delivery  systems. 
In  addition,  there  are  a  variety  of  software  packages  available  for  them.  The 
COS  II  software  currently  supports  only  a  videodisc  overlay  graphics  board  (by 
New  Media  Graphics),  although  support  for  conventional  640x400  boards  Is 
expected  later  In  1986. 

Upgrading  to  640x400x16  colors  resolution  would  Increase  the  cost  of  each 
delivery  system  by  approximately  $600-2,000.  Add-on  boards  currently  available 
for  IBM-compatible  PCs  are  listed  In  Table  2-8.  If  the  delivery  stations 
continue  using  the  AT4T  color  monitor,  which  currently  Interfaces  only  with 
their  proprietary  graphics  board,  then  these  higher  resolutions  cannot  be  made 
available.  In  that  situation,  the  only  recourse  Is  to  purchase  alternate 
monitors  that  can  be  driven  by  one  or  more  of  the  boards  listed. 

CONCLUSIONS/RECOMMENDATIONS: 

The  acquisition  of  the  new  release  of  COS  II  should  best  satisfy  the  need 
for  smaller  fonts.  The  new  release  will  provide  the  use  of  alternate  character 
sizes  when  authoring.  Also,  the  new  release  may  provide  a  means  to  access 
other  programs  that  will  display  “boilerplate”  graphics,  such  as  forms,  with 
the  monitor  In  the  higher  resolution,  monochrome  mode. 

2.3.3  Prototype  Configuration 

The  current  configuration  for  the  production  component  Is  three  ATCBT 
systems:  one  10-megabyte  hard  disk  system,  two  dual  floppy  systems,  one  modem, 
one  draft  quality  graphics  printer,  and  three  graphics  packages.  In  order  to 
meet  the  lesson  production  schedule,  this  configuration  must  be  expanded  and 
upgraded  to  extend  the  MDC  capabilities. 

2. 3. 3.1  Authoring  Stations 

ISSUE: 

The  number  of  authoring  stations  may  not  be  sufficient  to  meet  the 
command-wide  lesson  production  requirements. 

RESULTS: 

The  MDC  has  a  full-time  staff  of  six  authors  to  produce  and  validate  CBT 
lessons,  revise  CBT  lessons,  develop  graphics,  and  perform  data  management 
functions.  To  meet  the  command-wide  Implementation  schedule,  two  additional 
authoring  stations  and  one  additional  draft  quality  graphics  printer  are 
required.  One  station  configured  for  data  management  routines  and  connected  to 
a  modem  for  DDN  access  Is  sufficient  for  command-wide  Implementation. 


2-45 


Table  2-8 


High  Resolution  PC  Add-On  Boards 


Company 

Product 

Resolution 

Colors 

Display 

Memory 

(K) 

Price 

Array  Technology 

Graphics  Solution 

720x348 

16 

64 

$299 

Conographic 

Cono-color  40 

640x400 

16 

128 

$995 

Control  Systems 

Artist  Transformer 

640x480 

16-4096 

160 

$1795- 

1995 

Artist  II 

640x400 

16-4096 

128 

$1195- 

1395 

Emulex/Persyst 

BoB  Super  Display 
Adaptor 

640x400 

16 

16 

$525 

Eve rex 

Graphics  Edge 

640x400 

16 

4 

$499 

Frontier  Tech¬ 
nologies 

CADgraph-2 

640x480 

4-16 

256 

$995 

IBM 

Enhanced  Graphics 
Adaptor  (EGA) 

640x350 

64 

64 

$524 

Malnstreet 
Computer  Corp. 

Balance  II 

640x352 

4 

16 

$NA 

Number  Nine 

Revolution 

512x480 

256 

256 

$1995 

Paradise  Systems 

Modular  Graphics 
Card 

640x400 

16 

16 

$395 

Profit  Systems 

Multigraph  Plus 

640x400 

16 

128 

$595 

Sigma  Designs 

Color  400 

640x400 

16 

128 

$795 

Dazzler 

640x400 

16 

256 

$895 

Dazzler  2 

640x400 

16 

255 

$995 

Techmar 

Graphics  Master 

640x400 

16 

128 

$695 

Vectrix 

VX/PC 

672-480 

512 

384 

$2995 
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CONCLUS I ONS/RECOHMENDAT IONS: 


It  Is  reconmended  that  the  MDC  configuration  be  expanded  to  Include  five 
ATCBT  authoring  systems  and  two  draft  quality  graphics  printers  for  command¬ 
wide  Implementation.  Five  systems  will  also  provide  a  margin  of  safety  so 
lesson  production  and  maintenance  will  not  be  hampered  due  to  potential  system 
downtime. 


2. 3. 3. 2  Oata/FIle  Storage 

ISSUE: 

Authoring  on  floppy  diskettes  Is  time  consuming,  causes  a  backlog  of  disks 
and  Is  Impractical.  Use  of  the  hard  disk  configuration  seems  to  be  a  more 
efficient  use  of  file  storage  capability. 

RESULTS: 

The  Initial  design  of  the  authoring  stations  called  for  a  dual  floppy  disk 
system  with  a  virtual  disk  Installed.  During  the  evaluation.  It  was  discovered 
that  a  one-hour  lesson,  with  accompanying  graphics,  typically  consumes  up  to 
150K  of  memory,  which  Is  almost  half  of  the  320K  of  memory  normally  available 
on  a  single  floppy  disk.  However,  authoring  on  floppy  disks  caused  a  backlog 
of  file  copies  of  disks  and  difficulty  In  tracking  the  current  version  of  a 
lesson.  This  problem  will  only  Increase  as  the  number  of  lessons  and  graphics 
continues  to  Increase. 

CONCLUSIONS/RECOMMENDATIONS: 

All  authoring  stations  should  be  upgraded  to  at  least  a  10  megabyte  hard¬ 
disk  configuration,  If  not  a  21  megabyte  disk.  This  configuration  will 
eliminate  the  use  of  floppy  disks  as  the  primary  authoring  medium,  and  allow  a 
file  system  to  be  established  at  each  of  the  authoring  stations.  Floppy  disks 
could  then  be  used  only  to  provide  a  backup  of  current  work  in  case  of  power  or 
equipment  failure,  rather  than  as  the  primary  file  storage  facility.  Another 
option  that  could  be  investigated  for  quick  hard  disk  backup  is  a  tape  sorage 
back-up  system.  If  even  this  amount  of  disk  storage  becomes  a  problem  In  the 
future,  an  option  might  be  to  Install  external  hard  disk  drives. 

2. 3. 3. 3  Power  Disruptions 

ISSUE: 

The  MDC,  In  their  present  location  at  Travis  AFB,  have  been  subject  to 
power  outages  due  to  circuit  overloads  Internal  to  their  building  and  to 
external  power  line  downlngs  during  winter  storms. 

RESULTS: 

These  power  outages  have  significantly  Impacted  lesson  authoring  with 
estimates  of  up  to  one  man-week  of  work  lost  per  MDC  person.  Since  any  file 
that  was  open  (whether  lesson  or  graphic  or  test  data)  Is  lost  when  a  power 
outage  occurs,  with  an  additional  risk  of  hard  disk  crashes  and  subsequent 
damage,  the  potential  loss  warrants  attention. 


2-47 


There  are  Uninterruptable  Power  Supplies  (UPSs)  that  provide  frequency- 
stable,  constant  voltage  In  response  to  disturbances  on  Incoming  lines.  A  UPS 
consists  of  a  battery  charger,  batteries,  and  an  Inverter.  Some  UPSs  are  only 
activated  when  a  frequency  shift  or  a  blackout  occurs.  Others  require  that  the 
battery  charger  be  left  on.  Likewise,  some  systems  provide  no  break  or 
Interruption  In  the  power  supply.  Others  break  power  for  a  few  milliseconds 
when  transferring  to  the  battery  backup. 

The  price  for  a  500  VA  UPS,  which  should  provide  sufficient  power  for  a 
hard  disk  drive  PC  configuration,  ranges  from  $400.00  to  $1400.00.  Battery 
power  Is  typically  available  for  10  to  20  minutes,  which  should  be  ample  time 
to  finish  work,  save  files,  and  shut  down  the  system.  The  specific  operating 
characteristics  of  the  various  models  on  the  market  need  to  be  examined  further 
before  making  a  specific  purchase  suggestion. 

CONCLUSIONS/RECOHMENDATIONS: 

An  uninterruptable  power  supply  for  each  workstation  would  avert  the  lost 
time  and  potential  equipment  damage  due  to  power  blackouts. 

2.3.4  Lesson  Production,  Maintenance,  and  Management 

There  are  several  Issues  related  to  the  production,  maintenance,  and 
management  of  CBT  lessons  as  the  number  of  lessons  In  the  Inventory  Increases 
and  the  number  of  aerial  ports  online  Increases.  These  Issues  focus  on  the 
allocation  of  personnel  resources  at  the  MDC  and  the  development  of  procedures 
to  manage  the  CBT  lesson  Inventory. 

2. 3. 4.1  Lesson  Production  Schedule 

ISSUE: 

As  an  OPR,  the  MDC  staff  will  be  tasked  with  producing  a  substantial 
number  of  CBT  lessons  during  command-wide  implementation,  as  well  as 
maintaining  the  CBT  lessons  that  are  already  disseminated  to  the  field.  The 
requirement  to  collect  lesson  revision  data  and  update  lessons  on  a  quarterly 
basis  will  become  more  time  consuming  as  the  number  of  lessons  In  the  Inventory 
Increases. 

RESULTS: 

In  order  to  build  the  CBT  lesson  Inventory  for  command-wi de 
implementation,  ISO,  Inc.  will  be  tasked  to  develop  many  of  the  CBT  lessons  for 
the  inventory  during  the  follow-on  contract  period.  ISD,  Inc.  personnel  will 
design,  develop,  and  validate  lessons.  The  lessons  will  then  be  delivered  to 
the  MDC  for  Inclusion  In  the  Inventory  and  dissemination  to  the  field. 

CONCLUSIONS/RECOMMENOATIONS: 

To  minimize  the  Impact  on  MDC  personnel  resources,  ISD,  Inc.  will  utilize 
subject  matter  specialists  from  HQ  MAC  staff,  ATTC  instructors,  and  Travis 
aerial  port  squadron  supervisors  as  required  in  the  lesson  design  and  review 
process. 


2. 3. 4. 2  Personnel  Resources 

ISSUE: 

Given  the  current  HOC  staffing  level  and  tasking  and  the  requirements  for 
producing  and  maintaining  CBT  lessons  for  coninand>w1de  Implementation,  the  MDC 
personnel  resources  assigned  to  CBT  lesson  development  may  be  Insufficient  to 
meet  the  schedule. 

RESULTS: 

The  ATCBT  authoring  environment  was  designed  to  prompt  a  subject  matter 
specialist  through  the  authoring  process,  so  a  minimal  amount  of  training  would 
be  required.  ISO,  Inc.  will  provide  the  MDC  with  an  updated  authoring/editing 
user  guide  during  command-wide  Implementation.  This  could  be  used  to  train  the 
ATTC  instructors  In  CBT  lesson  design  and  development.  In  addition,  a  number 
of  the  ATTC  Instructors  participated  In  review  meetings  during  the  design  of 
the  ATCBT  authoring  environment  and  development  of  the  prototype  CBT  lessons 
and  In  the  training  workshops  conducted  at  the  MDC  during  the  evaluation 
period. 

CONCLUSIONS/RECOMMENDATIONS: 

Since  the  MDC  Is  co-located  with  the  ATTC  and  both  organizations  report  to 
the  1492  ATTF,  It  may  be  feasible  to  augment  the  MDC  authors  with  ATTC 
instructors  to  develop  CBT  lessons.  It  Is  also  recommended  that  three  delivery 
stations  be  Installed  In  the  ATTC  classrooms  to  provide  students  and 
Instructors  access  to  the  CBT  lessons. 

2. 3. 4. 3  Lesson  Management 

ISSUE: 

The  MDC  will  be  the  OPR  for  the  ATCBT  lesson  Inventory.  As  OPR,  they  will 
be  responsible  for  activities  such  as  assigning  lesson  designators,  monitoring 
status  of  lesson  development,  performing  quality  control,  maintaining  a  library 
of  graphics,  disseminating  new  lessons  to  the  field,  collecting  and 
consolidating  revision  comments,  analyzing  test  Item  data,  revising  lessons, 
and  sending  quarterly  lesson  updates.  In  addition,  the  MDC  will  be  responsible 
for  ensuring  that  all  of  the  consumables  (I.e.,  diskettes  and  mailers)  required 
to  support  the  program  are  available. 

RESULTS: 

Procedures  need  to  be  established  for  managing  the  CBT  lesson  Inventory. 
For  example,  lesson  updates  which  occur  In  response  to  comments  from  the  field, 
test  Item  analysis,  and/or  changes  In  subject  matter  need  to  be  edited  and 
disseminated  to  the  ATCBT  Installations.  In  fact,  the  "Introduction  to  PACS" 
lesson  developed  for  the  prototype  evaluation  currently  requires  an  update  to 
reflect  a  change  In  function  In  the  PACS  keyboard.  Likewise,  new  lessons, 
whether  they  are  MDC-developed  or  contractor-developed,  will  need  to  be  sent 
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out  to  installations.  Procedures  for  disseminating  CBT  lessons  need  to  include 
at  least  the  following  steps. 

1.  Copy  the  updated  or  new  lesson(s)  and  picture  files  onto  floppy 

disks. 

2.  Construct  a  batch  (.BAT)  file  that  will  cause  the  files  to  be  copied 
automatically  to  the  appropriate  subdirectory  at  the  delivery 
stations  so  that  ULC  monitors  need  not  know  MS-DOS  copy  and  delete 
commands. 

3.  Update  the  lesson  management  data  files  for  the  ULC  to  reflect  new 
1esson(s}. 

4.  Mail  the  copies  to  the  ULCs  with  a  distribution  letter  and 

installation  instructions. 

As  discussed  in  Section  2. 2. 2. 2,  there  is  a  requirement  for  an  ATCBT 

catalog  of  lessons  for  the  ULC  and  supervisors  in  the  field.  It  is  unclear 

whether  the  quarterly  updates  of  the  HQ  AFRES  Transportation  Training  Program 
Catalog  are  of  sufficient  detail  to  meet  the  requirements.  The  implications  of 
developing  a  catalog  based  on  the  lesson  specifications  prepared  for  each  CBT 
lesson  should  be  explored. 

The  MOC  staff  has  already  addressed  a  number  of  lesson  management  issues. 
They  are  using  a  ZlOO  to  support  some  of  the  recordkeeping  functions,  such  as 
assigning  lesson  designators,  recording  development  time,  establishing  in-house 
quality  control  procedures. 

CONCLUSIONS/RECOMMENOATIONS: 

It  is  recommended  that  the  MDC  develop  a  detailed  set  of  procedures  for 
managing  the  CBT  lesson  inventory.  ISD,  Inc.  will  provide  suggestions  and 
assistance  in  defining  procedures  as  required.  The  procedures  for  CBT  lesson 
management  need  to  be  incorporated  into  the  ATCBT  User  Guide. 
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SECTION  3.0 


COMMAND-WIDE  IMPLEMENTATION  PLAN 

This  section  presents  the  proposed  plan  for  command-wl de  Implementation  of 
the  ATCBT  system.  The  plan  Is  based  on  the  requirements  In  the  contract 
Statement  of  Work,  the  results  and  recommendations  of  the  prototype  ATCBT 
system  evaluation,  and  discussions  with  HQ  MAC  personnel.  It  represents  the 
best  efforts  of  ISD,  Inc.  to  specify  all  of  the  events  which  must  occur  and 
propose  a  tentative  schedule  to  accomplish  these  events.  Implementation  of  the 
plan  Is  subject  to  approval  by  HQ  MAC. 

The  proposed  schedule  Is  presented  In  the  first  subsection  to  serve  as  a 
framework  for  discussing  the  events  which  must  occur  for  command-wide 
Implementation.  The  discussion  of  events,  organized  by  the  three  components  of 
the  ATCBT  systems— delivery,  management,  and  production— follows. 

3.1  Proposed  Schedule 

The  scope  of  the  command-wide  Implementation  Is  to  Install  ATCBT  systems 
at  41  sites  worldwide  with  an  Inventory  of  approximately  200  hours  of  CBT 
Instruction  to  provide  Initial,  recurring,  and  upgrade  training  for  AFSC  605XX 
personnel.  The  schedule  for  command-wide  Implementation  Is  based  on  the  period 
of  performance  for  the  option  years  of  the  ISD,  Inc.  contract.  As  shown  In 
Figure  3-1,  the  projected  start  date  Is  1  April  1986.  This  date  Is  Intended  to 
allow  time  for  report  review  and  revisions,  approval  by  HQ  MAC,  and  Initiation 
of  subsequent  contract  delivery  orders  to  perform  the  Implementation  tasks. 
The  completion  date,  15  February  1988,  Is  the  end  date  for  the  ISD,  Inc. 
contract. 

The  schedule  In  Figure  3-1  Is  organized  by  the  events  for  each  of  the 
ATCBT  system  components.  As  shown,  there  are  a  number  of  concurrent  events 
across  components  which  must  be  completed  within  the  first  three  to  six  months. 
Some  of  these  events  are  In  preparation  for  command-wide  Implementation.  These 
Include  the  site  survey  and  the  development  of  the  Information  Systems 
Requirement  Document  shown  under  the  delivery  component  events.  Some  of  the 
events  are  required  to  Implement  revisions  resulting  from  the  operational  test 
and  evaluation  of  the  components.  These  Include  revisions  to  the  data 
management  routines,  expansion  of  the  CONTROL  software,  and  updates  to  the  user 
documentation/training  shown  under  the  management  component  events. 

To  ensure  that  the  revisions  are  Integrated  and  the  ATCBT  system  Is 
operationally  ready  for  command-wide  Implementation,  a  field  test  Is  planned 
for  months  4  and  5  (1  July  1986  to  31  August  1986).  After  the  field  test  is 
completed,  the  remaining  command-wide  Installations  are  scheduled  to  begin.  As 
shown  In  Figure  3-1,  seven  of  the  41  sites  will  be  Installed  at  the  end  of  the 
field  test  (31  August  1986)  and  all  41  sites  are  planned  for  Installation  by 
the  end  of  the  contract  period  (15  February  1988).  The  numbers  shown  In 
parentheses  on  the  timeline  for  worldwide  Implementation  represent  the  total 
number  of  site  Installations  planned  for  completion  at  that  point  In  the 
schedule.  As  shown,  11  are  planned  by  30  November  1986,  19  by  31  March  1987, 
26  by  30  June  1987,  33  by  30  September  1987,  36  by  31  November  1987,  with  the 
remaining  five  completed  by  15  February  1988.  Section  3.2.4  provides  the 
detailed  Implementation  plan. 
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Figure  3-1.  Schedule  for  C<MMnd-uide  iMplewentation 


In  order  to  support  the  CBT  lesson  production  and  maintenance  requirements 
for  command-wide  implementation,  the  hardware/software  configuration  at  the  MDC 
needs  to  be  expanded  and  upgraded  and  the  authoring  environment  needs  to  be 
enhanced.  A  period  of  six  months  (1  April  1986  through  30  September  1986)  is 
scheduled  for  the  MDC  hardware/software  configuration  upgrade. 

A  period  of  six  months  (1  April  1986  through  30  September  1986)  is 
scheduled  for  enhancements  to  the  authoring  environment.  Of  course,  this 
schedule  assumes  that  the  software  upgrades  are  approved.  As  shown  in  the 
figure,  this  event  will  be  performed  concurrently  with  the  initial  CBT  lesson 
production  and  maintenance  events.  A  number  of  enhancements  to  the  current 
authoring  prototype  have  already  been  identified,  but  more  will  surface  as  new 
lessons  are  being  developed.  It  is  anticipated  that  most  of  the  improvements 
will  be  identified  and  can  be  implemented  during  this  six-month  period. 

To  meet  the  objective  of  producing  and  disseminating  approximately  200 
hours  of  CBT  instruction  during  command-wide  implementation,  a  joint  CBT  lesson 
production  effort  is  required.  As  shown  in  Figure  3-1,  both  the  MIC  and  ISO, 
Inc.  are  scheduled  to  develop  lessons  concurrently  during  the  implementation 
period.  Phase  I  in  the  schedule  represents  the  target  of  having  approximately 
60  hours  of  CBT  instruction  in  the  field  by  30  September  1986.  Phase  II  in  the 
schedule  represents  the  period  for  producing  .he  approximately  140  hours  of 
instruction  that  remain.  As  shown,  CBT  lesson  maintenance  and  management  is  an 
ongoing  MIC  activity  beginning  with  command-wide  implementation. 

3.2  Delivery  Component 

There  are  four  events  scheduled  for  command-wide  implementation  of  the 
delivery  component:  (1)  a  survey  of  all  the  installation  sites,  (2)  the 
development  of  the  Information  Systems  Requirement  Document,  (3)  the  ATCBT 
field  test,  and  (4)  worldwide  implementation.  Each  of  these  events  is 
described  in  detail  below. 

3.2.1  Site  Survey 

An  ATCBT  survey  must  be  disseminated  to  all  of  the  command-wide 
implementation  sites  within  the  first  ten  days  of  contract  start.  The  purpose 
of  the  survey  is  to:  (1)  solicit  specific  information  about  the  installation 
site,  (2)  specify  the  facilities  preparation  which  must  be  accomplished  at  the 
site  prior  to  installation,  and  (3)  define  the  maintenance  responsibilities. 

The  information  to  be  solicited  from  the  sites  includes,  but  is  not 
limited  to: 

1.  Work  Force  Description.  Total  number  of  personnel  on  site  requiring 
AFSC  605XX  training,  broken  down  by  shifts  and  work  centers. 

2.  Work  Center  Locations.  Identification  of  specific  buildings  and  work 
centers  within  buildings  where  an  ATCBT  delivery  station  is  required. 

3.  Communications  Systems  Requirements.  Type  of  system,  type  of  lines, 
responsible  organization  for  on-base  system. 

4.  Specific  Points  of  Contact.  Name,  title,  and  phone  number  for 
personnel  responsible  for  ATCBT  implementation. 
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5.  Tentative  Schedule.  Time  constraints  (I.e.,  Inspections  or 

exercises)  that  might  Impact  Implementation  schedule. 

The  facilities  preparation  Information  to  be  provided  to  the  sites 
Includes,  but  Is  not  limited  to: 

1.  General  Communication  Requirements.  Class  C  lines  are  needed  to  send 
data,  using  dial-up  modems,  from  several  remote  points  on  base  to  the 
ULCs.  The  Class  C  lines  should  meet  standard  private  line 
requirements  for  voice  grade  circuits.  These  requirements  are: 

0  C  message  noise  should  be  less  than  31  dB  RNC  at  the  zero  data 
level  point 

0  Impulse  noise  at  no  more  than  one  hit  In  one  minute  at  the  zero 
data  test  level 

0  Signal-to-nolse  ratio  at  about  -45  dB  for  modem  receive 
sensitivity 

0  RJ-11  modular  telephone  Jack  connector 

If  these  requirements  cannot  be  met  by  Class  C  lines,  then  dedicated 
lines  may  be  used.  Specific  requirements  will  be  issued  for  each 
site  as  locations  for  terminals  are  identified. 

2.  General  Power  Requirements.  Grounded  outlets  should  be  provided  at 
each  work  station  where  terminals  will  be  placed.  The  area  should  be 
Isolated  as  much  as  possible  from  electrical  noise  and  devices  which 
cause  excessive  voltage  level  variations.  Care  should  be  taken  to 
eliminate  areas  which  are  not  rated  for  the  power  use  required.  For 
non-CONUS  sites,  transformers  to  allow  110  VAC,  60  Hz  operation 
should  be  provided  by  the  squadron.  One  transformer  per  terminal 
should  be  provided  with  a  rating  of  2000  watts  or  higher. 

3.  Space  Requirements/Environmental  Factors.  The  ATCBT  computer  should 
be  placed  In  a  relatively  dust- free  place.  Airborne  dust,  dirt,  and 
smoke  can  cause  excess  wear  on  moving  surfaces,  short  circuits 
(especially  In  the  presence  of  high  humidity),  and  cause  read/write 
errors  on  the  disk.  A  cleaning  kit  needs  to  be  available  for  use  at 
each  site. 

The  ATCBT  computer  should  be  placed  away  from  heat  and  direct 
sunlight.  Unusually  high  temperatures  coupled  with  low  humidity  can 
cause  static  problems.  Normal  temperature  operating  range  Is  10  to 
40  degrees  centrl grade  with  humidity  variations  of  10  to  95*. 

The  front  and  rear  panels  of  the  ATCBT  computer  should  not  be 
obstructed.  The  computer  Is  cooled  by  a  fan  located  on  the  rear 
panel.  This  fan  pulls  air  In  through  air  Inlets  on  the  base  of  the 
system.  The  base  of  the  system  should  be  kept  clear  of  papers  or 
other  materials  that  could  obstruct  air  flow. 


The  system  should  not  be  placed  In  a  small  enclosed  area.  The  system 
dissipates  140  watts  (477.7  BTU/hour)  which  may  be  enough  to  raise 

the  temperature  beyond  the  maximum  allowed  If  enclosed  in  a  small 

area.  This  will  also  decrease  the  air  flow  around  the  system. 

Each  work  station  needs  space  for  a  standard  desk  or  table-size  desk, 
or  a  learning  carrel,  on  which  to  place  the  terminal  and  still 

provide  working  room.  An  area  for  adjunct  learning  materials  Is 

needed  at  each  ATCBT  work  station. 

4.  Support  Materials.  The  following  Items  should  be  available:  anti¬ 
static  mats  for  work  stations  with  nylon  carpet,  surge  protectors  for 
each  station  (CONUS  only),  security  measures  as  deemed  appropriate, 
cleaning  kits,  small  portable  vacuum,  printer  paper  and  ribbons, 
floppy  disks,  floppy  disk  storage  files. 

The  maintenance  responsibllties  need  to  be  defined  for  the  sites  In  terms 
of  the  following: 

1.  Each  site  has  ownership  of  the  training  systems. 

2.  Support  available  from  HQ  MAC  and  other  support  organizations. 

3.  Explanation  of  warranty. 

4.  Description  of  preventive  and  corrective  maintenance  responsibllties. 

5.  Procedures  for  Form  9's. 

6.  Specifications  for  local  maintenance  support. 

3.2.2  Information  Systems  Requirement  Document 

Based  on  the  results  of  system  evaluation  presented  in  Section  2.1,  the 
minimum  requirements  for  the  ATCBT  system  are  listed  below.  Since  the  ATCBT 
software  (including  the  delivery,  management,  and  production  components)  Is 
transportable  across  a  family  of  PC  compatible  microcomputers.  It  does  not 
matter  which  system  Is  procured. 

Delivery  Station  Characteristics: 

0  MS-DOS  version  3.1  or  later 

0  Color  graphics  monitor  (320x200  resolution) 

0  Color  board 

0  640K  RAM 

0  21  megabyte  hard  disk  drive  with  one  floppy  disk  drive 

0  Keyboard 

0  Parallel  port 

0  Serial  port 

0  At  least  three  additional  slots  for  internal  modem,  over  board,  and 

other  add-ons 
0  Calendar  clock 

Modem: 

0  CCITT  protocol  for  non-CONUS 

0  Bell  212  for  CONUS 

0  Blind  dialing  (does  not  wait  for  dial  tone)  for  non-CONUS 

0  Receive  sensitivity  at  least  -45  dBm 

0  Transmit  sensitivity  at  -10  dBm 
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0  Automatic  answer  setting 

0  DTR  always  on  setting 

0  IBM  PC  compatible 

0  RJ-11  Jack  connector 

The  plan  is  to  procure  the  hardware  for  command-wide  implementation  under 
the  Air  Force  standard  microcomputer  buy.  Since  the  system  available  under  the 
standard  buy  is  an  IBM  AT  clone  rather  than  an  IBM  PC  clone,  it  meets  or 
exceeds  the  minimum  requirements  for  the  ATCBT  system. 

3.2.3  ATCBT  Field  Test 

The  purpose  of  the  prototype  evaluation  was  to  field  test  the  design  of 
the  ATCBT  system  components.  The  purpose  of  the  field  test  is  to  ensure  that 
all  of  the  system  revisions  and  enhancements  identified  as  a  result  of  the 
prototype  evaluation  are  integrated  and  operationally  ready  for  command-wide 
implementation.  A  period  of  two  months  is  scheduled  for  the  field  test  in 
order  to  shake  out  system  “bugs  and  glitches"  in  an  operational  environment. 
In  this  way,  any  fixes  to  the  revised  software  and  changes  to  the  updated  user 
support  documentation  can  be  incorporated  before  worldwide  implementation 
begins. 

The  field  test  is  scheduled  to  begin  90  days  after  contract  start  to  allow 
sufficient  time  to  implement  and  test  the  software  revisions  for  the  management 
component,  document  procedures  and  prepare  training/job  aids,  provide  lead  time 
to  schedule  hardware  delivery,  and  prepare  facilities  at  the  test  sites. 

The  seven  sites  listed  below  are  scheduled  for  the  field  test. 

1.  1492  ATTF  (MDC  and  ATTC) 

2.  Travis  AFB,  CA 

3.  Oakland  International  Airport,  CA 

4.  San  Francisco  International  Airport,  CA 

5.  Los  Angeles  International  Airport,  CA 

6.  Norton  AFB,  CA 

7.  Dyess  AFB,  TX 

The  field  test  also  supports  the  events  occurring  in  the  production 

component  during  the  first  six  months.  It  provides  the  opportunity  for  the  MDC 
to  go  into  a  full-scale  CBT  lesson  production/maintenance/management  mode. 
Procedures  for  lesson  dissemination,  revision,  and  updating  can  be  tested  and 
documented.  In  addition,  approximately  50  to  60  hours  of  CBT  instruction  will 
be  completed  by  the  end  of  the  field  test.  This  provides  a  substantial  CBT 
lesson  inventory  at  the  start  of  worldwide  implementation. 

3.2.4  Worldwide  Implementation 

The  strategy  for  worldwide  implementation  is  to  cluster  sites  by 
geographic  location  to  facilitate  scheduling  and  minimize  travel  costs.  The 
site  installation  team  consists  of  ISO,  Inc.  and  HQ  MAC  personnel.  The  size  of 
the  installation  (number  of  stations)  determines  the  number  of  personnel  and 
the  time  required  at  each  site.  For  planning  purposes,  one  week  per  site  and 
three  to  six  personnel  per  cluster  (depending  on  the  number  of  sites)  is 
estimated. 
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Table  3-1  presents  a  tentative  schedule  for  the  41  sites  to  be  Implemented 
worldwide.  It  lists  the  sites  by  cluster  In  the  proposed  order  of 
Installation.  Tentative  dates  are  Identified  for  each  cluster,  as  well  as  an 
estimate  of  the  number  of  stations  for  each  site.  Both  the  order  of  the 
clusters  and  the  sites  within  each  cluster  are  subject  to  approval  by  HQ  MAC. 

As  shown.  Cluster  1  represents  the  sites  selected  for  the  field  test. 
This  cluster  Includes  two  of  the  four  prototype  evaluation  sites.  Clusters  2 
and  3  Include  the  remaining  two  prototype  evaluation  sites.  Since  the  standard 
buy  microcomputer  will  be  procured  for  command-wide  Implementation,  the 
decision  was  made  to  replace  the  AT&T  6300  computers  currently  In  the  field 
early  In  the  schedule.  This  will  standardize  the  field  configuration  and 
minimize  maintenance  requirements.  The  plan  Is  to  upgrade  the  AT&T  6300 
computers  as  required  and  use  them  for  authoring  and  as  stand-alone  stations. 

The  tentative  dates  are  subject  to  change  due  to  a  variety  of  factors. 
These  Include,  but  are  not  limited  to,  lead  time  for  hardware  procurement, 
vendor  delivery  schedule  delays,  lead  time  for  site  preparation,  completion  of 
base  conmunlcatlon  requirements,  and  non-aval lability  of  site  personnel  due  to 
Inspections  or  exercises.  An  Interval  of  two  to  four  weeks  Is  planned  between 
each  cluster  to  allow  scheduling  flexibility.  Also,  some  flexibility  needs  to 
be  provided  to  allow  time  to  revisit  a  site  within  a  cluster  If  unique, 
unanticipated  operational  problems  occur. 

Table  3-1  also  shows  the  estimated  number  of  stations  per  site.  The 
number  of  stations  Is  for  planning  purposes  only.  The  actual  number  of 
stations  will  be  determined  on  the  basis  of  the  site  survey  to  be  distributed 
for  command-wide  Implementation.  The  number  of  stations  was  determined  by  a 
three-step  process: 

1.  Identify  the  number  of  separate  work  stations  at  each  site. 

2.  Determine  the  number  of  605XXs  at  each  separate  work  station. 

3.  Decide  how  many  terminals  should  be  located  at  each  separate  work 
station. 

This  process  takes  Into  account  remote  work  station  locations,  as  well  as 
total  throughput.  In  order  to  preclude  travel  by  potential  users  to  the  primary 
work  centers.  The  determination  of  size  and  work  stations  was  based  on  an 
Initial  survey  conducted  In  January  1985.  For  those  aerial  ports  for  which  no 
surveys  were  received,  numbers  were  extrapolated  from  sImlUr  sized  aerial 
ports  with  similar  missions.  Total  throughput  was  used  to  determine  additional 
delivery  stations.  In  that  more  than  126  605XXs  at  any  one  work  station  during 
any  one  shift  would  require  two  terminals.  (This  does  not  account  for  AFRES 
personnel  who  are  potentially  additional  systems  users.)  This  cutoff  figure 
was  based  on  the  following  four  assumptions: 
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Tentative  Installation  Schedule  for 
Command- Wide  Implementation 


Cluster 

Sequence 

Sites  in  Cluster 

Estimated 
Number 
of  Stations 

Tentative 

Dates 

1 

1492  ATTF  (includes  MDC  and  AHC) 

8 

Field  Test 

Travis  AFB,  CA 

6 

01  Jul  86 

Oakland  International  Airport,  CA 

1 

31  Aug  86 

San  Francisco  International  Airport,  CA 

1 

Los  Angeles  International  Airport,  CA 

1 

Norton  AFB,  CA 

6 

Dyess  AFB,  TX 

3 

2 

Charleston  AFB.  SC 

7 

08  Sep  36 

Pope  AFB,  NC 

4 

26  Sep  86 

3 

Rhein  Main  AB,  Germany  (includes  PRC) 

8 

22  Sep  86 

Ramstein  AB,  Germany 

5 

10  Oct  86 

4 

McGuire  AFB,  NJ 

5 

12  Jan  87 

John  F.  Kennedy  International  Airport,  NY 

1 

30  Jan  87 

Philadelphia  International  Airport,  PA 

1 

Dover  AFB,  DE 

6 

NAS  Norfolk,  VA 

2 

Andrews  AFB,  MO 

3 

5 

Little  Rock  AFB,  AR 

5 

16  Feb  87 

Altus  AFB,  OK 

3 

06  Mar  87 

6 

McChord  AFB,  WA 

5 

06  Apr  87 

Elmendorf  AFB,  AK 

5 

01  May  87 

Ellison  AFB,  AK 

1 

Hickam  AFB,  HI 

5 

7 

Scott  AFB,  11  (includes  PRC) 

4 

18  May  87 

St.  Louis  International  Airport,  MO 

1 

05  Jun  87 

Howard  AFB,  Panama 

5 

8 

RAF  Mildenhall,  UK  (includes  5  MAPS  and 

313  APS) 

5 

06  Jul  87 

Prestwick,  Scotland 

2 

31  Jul  87 

Torre jon,  Spain 

2 

Rota,  Spain 

2 

Table  3-1  (Continued) 


Cluster 

Sequence 

Sites  in  Cluster 

Estimated 
Number 
of  Stations 

Tentative 

Dates 

9 

Aviano  AB,  Italy 

2 

31  Aug  87 

NAS  Sigonella,  Italy 

2 

18  Sep  87 

Naples,  Italy 

2 

10 

Incirlik,  Turkey 

2 

19  Oct  87 

Hellenikun  AB,  Greece 

2 

13  Nov  87 

Lajes  Field,  Azores 

2 

11 

Kadena  AB,  Japan 

6 

04  Jan  88 

Clark  AB,  Phillipines 

6 

31  Jan  88 

Anderson  AFB,  Guam 

4 

Yokota  AB,  Japan 

6 

Osan  AB,  Korea 

5 

152 

1.  Each  person  takes  at  least  one  lesson  a  week. 

2.  There  would  be  75%  usage  at  any  one  terminal  with  25X  of  total 
availability  used  for  transition  time  and  for  downtime. 

3.  Therefore,  over  a  24-hour  period,  18  people  would  be  able  to  take  one 
one-hour  lesson. 

4.  The  stations  would  be  continuously  used  seven  days  a  week. 

The  conclusion  is  that  126  people  could  take  one  lesson  a  week  on  any  one 
delivery  station.  The  most  critical  factor  in  determining  the  required  number 
of  delivery  stations  prpved  to  be  location  of  the  work  station.  Many  stations 
are  physically  removed  from  each  other,  thus  the  implication  is  that  each 
station  needs  their  own  microcomputer. 

3.3  Management  Component 

Three  events  are  scheduled  for  command-wide  implementation  of  the 
management  component:  (1)  revisions  to  the  the  data  management  routines,  (2) 
expansion  of  the  CONTROL  software,  and  (3)  updates  to  the  user  documentation 
and  training  aids.  Each  of  these  events  is  described  in  detail  below. 


3.3.1  Data  Management  Revisions 

As  described  In  Section  2.2.1,  a  number  of  revisions  need  to  be 
Incorporated  Into  the  data  management  routines.  These  Include  major  revisions 
to  the  training  management  reports  to  make  them  more  useful  to  the  OJT 
supervisors,  section  supervisors,  and  ULC  monitors.  They  also  Include 
simplifying  the  procedures  for  ULC  monitors  to  Input  database  Information  and 
generate  reports.  Based  on  the  recommendations  from  the  prototype  evaluations, 
the  following  changes  are  planned  for  the  training  Information  management 
reports: 

1.  Revise  the  completion  report  to  provide  a  record  of  the  trainee 
progress  through  the  parts  of  a  lesson,  Including  segment  and  lesson 
exam  scores  and  dates.  Delete  the  report  to  the  student. 

2.  Provide  a  list  of  personnel  currently  registered  on  the  ATCBT  system. 

3.  Correct  the  problems  encountered  in  the  CBT  utilization  data  report. 

4.  Reconsider  the  use  of  the  list  of  students  which  shows  no  activity  In 
the  system  for  a  specified  period  of  time. 

5.  Provide  a  history  file  for  each  student  registered  on  the  system. 

The  requirement  for  an  on-line  query  capability  at  the  work  station  needs 
to  be  further  defined  in  order  to  determine  the  scope  of  the  revisions. 
Likewise,  the  capability  for  supervisors  to  Input  lesson  revision  comments  on¬ 
line  needs  to  be  reviewed  to  determine  its  impact  during  command-wide 
implementation. 

Once  the  requirements  are  Identified,  the  Interfaces  and  procedures  for 
inputting  database  information  and  generating  reports  can  be  simplified.  The 
format  for  the  data  can  be  defined  more  clearly  for  faster  entry  and  the 
processing  procedures  can  be  streamlined  and  made  more  specific.  The  goal  is 
to  make  the  ULC  monitor  data  management  tasks  as  efficient  as  possible  through 
software  enhancements.  The  requirement  to  document  the  revisions  Is  discussed 
in  Section  3.3.3  below. 

3.3.2  CONTROL  Software  Expansion 

The  ATCBT  CONTROL  software  (an  ISD,  Inc.  product)  operates  "underneath" 
the  MS-DOS  software  (a  Microsoft  Corp.  product).  CONTROL  controls  the  ATCBT 
events  that  occur.  It  resides  as  a  monitor  to  tell  CDS  (a  product  of  EIS, 
Inc.)  when  to  run  a  lesson;  it  moves  files  from  hard  disk  to  virtual  disk  to 
executable  RAM,  determines  when  and  where  to  get  the  lessons,  determines  when 
and  where  to  send  data,  determines  the  status  of  communication  lines,  allows 
administrative  actions  (protected  from  students)  for  data  management  actions, 
and  Issues  commands  to  MS-DOS. 

Currently,  CONTROL  is  designed  to  handle  commercial  communication  lines. 
Class  A  lines.  Class  C  lines,  and  null  modems.  Expansion  of  its  capabilities 
to  handle  dedicated  lines  and  integrated  digital  lines  is  needed  to  support 
command-wide  implementation  in  cases  where  those  specialized  lines  are 
required.  It  is  planned  to  (1)  incorporate  more  error  correction  to  accomodate 
noisy  lines  on  bases,  (2)  provide  an  on-line  testing  monitor  to  check  line 


quality,  and  (3)  enhance  the  interface  to  users  for  setting  communication 
parameters  and  configuring  the  system.  Other  expansions  which  may  be  necessary 
include  support  for  a  data  query  from  supervisors  at  remote  terminals. 

3.3.3  User  Oocumentation/Training  Aids 

The  ATCBT  User  Guide  must  be  updated  to  reflect  the  revisions  in  process 
and  procedures  described  in  Section  2.2.  These  include  the  operating 
instructions  and  procedures  developed  and  validated  using  the  prototype 
evaluation,  as  well  as  the  modifications  to  be  made  prior  to  and  during  the 
field  test.  The  revised  ATCBT  User  Guide  needs  to  (1)  determine  the  format  and 
use  of  the  ATCBT  student  history  file  (i.e.,  printed  reports  included  in 
training  records,  out-processing  documents  sent  downline  to  next  duty  station, 
etc.),  (2)  establish  the  policy  for  retaining  training  data  at  the  ULC,  (3) 
establish  ownership  of  the  equipment  and  specify  the  maintenance  management 
policy  (i.e.,  can  Air  Force-wide  small  computer  maintenance  contracts  be  used 
or  do  local  on-call  agreements  need  to  be  made?),  (4)  provide  guidelines  (as 
dictated  by  HQ  MAC)  on  the  availability  of  organizational  resources  at  the 
squadron,  base.  Numbered  Air  Force,  1492  ATTF,  and  HQ  MAC  levels,  as 

appropriate,  (5)  detail  the  procedures  for  the  MDC  and  the  ULC  for  lesson 

dissemination,  maintenance,  and  management. 

Based  on  the  results  of  the  prototype  evaluation,  a  variety  of  easy-to-use 
training/job  aids  need  to  be  implemented  for  the  ATCBT  system  users.  These  Job 
aids  need  to  extend  beyond  the  ULC  monitors  to  the  OJT  and  section  supervisors, 
the  trainees,  and  to  the  MOC  in  their  role  of  providing  operational  support  to 
the  field.  They  range  from  simple  instructions  on  use  of  the  keyboard  placed 
on  the  training  terminal  at  the  work  station,  to  a  list  of  adjunct  materials 
required  to  support  the  CBT  lessons  in  the  inventory,  to  the  step-by-step 

procedures  for  reformatting  the  hard  disk  to  recover  from  crashes. 

3.4  Production  Component 

The  events  scheduled  for  command-wide  implementation  of  the  production 
component  are:  (1)  MOC  configuration  upgrades,  (2)  enhancements  to  the 

authoring  environment,  and  (3)  full-scale  CBT  lesson  production  and 
maintenance.  Each  of  these  events  is  described  in  detail  below. 

3.4.1  MDC  Configuration  Upgrade 

The  proposed  configuration  for  the  MDC  production  component  is  listed 
below.  Paragraphs  from  Section  2.3  are  referenced  in  parentheses  after  each 
item  to  reiterate  the  need  for  the  upgrade. 

0  Five  (5)  authoring  stations  (2.3.3. 1  and  2. 3. 3. 2) 

0  Two  (2)  draft  quality  printers  (2.3. 3.1) 

0  Five  (5)  uninterruptable  power  supplies  (2. 3. 3. 3) 

0  Three  (3)  copies  of  Dr.  Halo  II  graphics  software  with  Summagraphics 
MM  series  digitizer  tablet  and  input  devices  (2. 3. 2. 2) 

0  One  (1)  PC  Eye  digitizer  system  (2. 3. 2. 2) 


0  Five  (5)  copies  of  latest  version  of  CDS  II  authoring  software 
(2. 3. 1.4  and  2. 3. 2. 3) 

0  Five  (5)  copies  of  CDSX  run-time  software  (2. 3. 1.5) 

0  Five  (5)  copies  of  CONTROL  software  (2.3. 1.5) 

The  software  upgrades  need  to  be  Implemented  as  quickly  as  possible.  This 
Is  to  minimize  the  requirement  to  convert  CBT  lessons  developed  using  the 
current  version  of  COS  II  authoring  software  and  graphics  development  software 
when  upgrade  does  occur.  Also,  the  proposed  enhancements  to  Improve  the 
efficiency  of  the  authoring  environment  cannot  be  started  until  the  software 
upgrades  are  In  place.  The  authoring  software  upgrades  are  commercially 
available;  thus,  minimal  lead  time  Is  required.  The  hardware  upgrade  may 
require  60  to  120  days  lead  time  for  Installation;  thus,  a  period  of  six  months 
Is  scheduled.  However,  the  sooner  the  hardware  upgrade  can  be  accomplished, 
the  easier  it  will  be  for  the  MDC  staff  to  meet  their  production  requirements. 

3.4.2  Enhancements  to  the  Authoring  Environment 

As  discussed  In  Section  2.3.1,  the  authoring  prototype  can  be  enhanced  In 
several  ways.  The  modifications  to  the  prompts  In  the  current  prototype  should 
be  Implemented  Immediately.  Once  the  upgrade  to  the  CDS  II  software  has  been 
Implemented  at  the  MDC,  a  number  of  features  can  be  added  to  the  authoring 
prototype.  These  Include  running  programs  external  to  CDS  II,  use  of  graphic 
commands  within  CDS  II,  access  to  alternate  text  sizes,  and  screen-based  text 
entry.  Furthermore,  additional  prototypes  using  these  new  features  can  be 
developed  for  specialized  topic  areas,  such  as  PACS  screen  masks  and  ADAM  III 
displays.  The  Production  Component  User  Guide  must  also  be  updated  to  reflect 
the  modifications  and  enhancements  to  the  authoring  prototype. 

3.4.3  CBT  Lesson  Production  and  Maintenance 

As  described  earlier,  the  production  of  approximately  200  hours  of  CBT 
Instruction  Is  a  joint  effort  between  the  MDC  and  ISD,  Inc.  The  priority  for 
lesson  topics  to  be  developed  for  command-wide  Implementation  was  established 
by  HQ  MAC  and  the  MDC.  Top  priority  was  given  to  those  STS  items  for  which  no 
formal  training  material  (I.e.,  slide/tape,  programmed  text)  currently  exists. 
The  STS  items  for  which  training  materials  already  exist  were  given  a  lower 
priority  for  development.  The  assignment  of  lesson  topics  for  contractor 
development  was  determined  by  HQ  MAC  and  the  MDC.  The  MDC  staff  Is  responsible 
for  developing  approximately  72  hours  of  Instruction,  while  ISD,  Inc.  is  tasked 
with  developing  approximately  120  hours  of  Instruction.  The  MDC  staff  Is  also 
responsible  for  maintaining  the  nine  prototype  lessons,  as  well  as  all  of  the 
new  lessons  as  they  are  added  to  the  Inventory. 

The  schedule  for  producing  the  lessons  Is  divided  Into  two  phases.  By  the 
end  of  Phase  I,  approximately  60  hours  of  CBT  Instruction  will  be  In  the 
Inventory.  This  Includes  20  to  24  hours  developed  by  the  MDC  and  30  hours 
developed  by  ISD,  Inc.,  plus  the  nine  hours  for  the  prototype  lessons.  The 
lesson  topics  to  be  developed  by  the  WC  during  Phase  I  are  listed  In  order  of 
priority  In  Table  3-2.  The  lesson  topics  to  be  developed  by  ISD,  Inc.  during 
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Phase  I  are  listed  In  order  of  priority  in  Table  3-3.  The  remaining  hours  of 
CBT  instruction  (approximately  140)  will  be  developed  during  Phase  II.  Table 
3-4  lists  the  lesson  topics  to  be  developed  by  the  MOC.  while  Table  3-5  lists 
the  lesson  topics  to  be  developed  by  ISO,  Inc.  during  this  period. 

Phase  II  lessonware  production  by  ISO,  Inc.  is  subject  to  further 
definition  and  verification  by  HQ  MAC.  The  emphasis,  priority,  and  actual 
topics  for  CBT  production  can  change  due  to  a  number  of  circumstances, 
including  HQ  MAC'S  response  to  comments  from  the  field  regarding  lessons  that 
are  desired/needed  for  OJT. 

The  number  of  lessons  scheduled  for  MDC  production  in  Phase  II  has  been 
reduced  due  to  the  increased  requirements  for  lesson  maintenance.  With  60 
hours  of  instruction  in  the  field  and  more  being  developed,  the  amount  of  time 
required  by  the  MDC  staff  to  maintain  and  manage  the  CBT  lesson  inventory  will 
increase  significantly. 
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Table  3-2 


Topics  for  Phase  I 
CBT  Development  by  the  MOC 


Related 

Estimated 

Lesson  Topics 

STS 

Items 

Hours 

Antl-HI jackIng/Termlnal  Security 

60530: 

12 

1.0 

Inflight  Meal  Procedures 

60530; 

9f 

1.0 

Introduction  to  ADAM  III 

60531: 

15a,  b,  9 

1.0 

Joint  Inspection,  DO  Form  2133 

60531; 

lOn 

2.0 

Determine  Passenger  Eligibility 

60530: 

8b(l) 

3.0 

Issue  and  Account  for  Supplies 
and  Equipment 

60531; 

13b(l)-(3) 

1.0 

Mobility  Work  Centers 

60531; 

7a,  b 

4.0 

60530; 

7a,  b 

Maintain  Mission  Folder,  Logs, 

Trip  Set-Up  Sheets,  and  Record 

Work  Center  Activities 

60531; 

lid 

1.0 

Flight  Schedules  and  Updated 

Messages 

60531; 

11a 

1.0 

M-Serles  Vehicles  (1  hour/vehicle) 

60531; 

8a(8),  8b(8) 

5.0 

Process  Unique  Passengers 

60530; 

91 

2.0 

TOTAL  22.0 

Table  3-3 


I 


Topics  for  Phase  I 
CBT  Development  by  ISO,  Inc. 


Related 

Estimated 

Lesson  Topics 

STS  Items 

Hours 

PACS  Processing 

i 

Processing  1 

60530;  8a(I)(c),  8b 

2.0 

Processing  2 

60530;  8b,  9a,  9e 

3.0 

Processing  3 

60530;  9d,  10b,  8a(l)(c), 
lOa(l),  14 

3.0 

Processing  4 

60530;  14 

1.0 

w  J 

Center  of  Balance  Markings  and 
Computations 

60531;  lOi(l)  (2) 

2.0 

Calculate  Placement  of  Cargo 

k'. 

in  Aircraft  and  Complete 

Actual  Weight  and  Balance 

Forms 

60531;  lli(l)  (2) 

4.0 

Process  Passenger,  Travel 
Restrictions,  Border  Clearance 

60530;  9a,  b 

2.0 

1 

Wide  Body  Cargo  Loader 

60531;  8a(9),  8b(9) 

4.0 

1 

Cash  Collection  Procedures 

60530;  9d 

2.0 

ADAM  III 

■ 

Air  Terminal  Cargo  Processing 

60531;  lOf(l)  (2) 

2.0 

Process  Surface  Arriving  Cargo 

60531;  10 

1.0 

Load  Planning 

60531;  Ilf,  h,  j 

1.0 

Cargo  Processing 

60531;  10 

2.0 

Cargo  Tracing  and  Visibility 

60531;  9c(4)  (5),  llg 

1.0 

- 

TOTAL  30.0 

li 
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Table  3-4 


Topics  for  Phase  II 
CBT  Development  by  the  MDC 


Lesson  Topics 

Related 

STS  Items 

■ 

Estimated 

Hours 

Fleet  Service  Equipment 
(1  hour/vehicle) 

60531;  13c(l)-(3) 

3.0  “ 

Process  Cargo  and  Mall  (Manifest) 

60531;  lOf(l).  (2) 

1.0 

Inspect  Shipments  for  General 
Markings 

60531;  lOa(l) 

1.0 

Frustrate  Air  Cargo  Shipments 

60531;  lOb(l) 

1.0 

Frustrate  Mall  Shipments 

60531;  lOb(l) 

1.0 

Maintain  Passenger  Listings 
(Flight  Setup) 

60530;  8b(2) 

3.0  1 

Select  Passengers  for  Movement 

60530;  8b(3)-(5) 

2.0 

Security  for  Cargo  and  Mall 

60531;  10f(3) 

1.0 

Maintain  Logs 

60531;  10f(4),  (6) 

1.0  -  “ 

On/Offload  Using  Aircraft  Engine 
Running  Procedures 

60531;  12f 

1.0 

Process  Passenger  Baggage 

60530;  9g 

4.0  1 

Store  Shipments  In  Appropriate 

Place 

60531;  lOg 

2.0 

Re-Ice  Shipment  and  Annotate 

60531;  10f(5) 

1.0 

Documentation  for  Palletized  Air 
Shipments 

60531;  10k(l)-(3) 

1.0  * 

Passenger  Bus 

60530;  lla(3),  llb(3) 

1.0 

Maintain  On-Hand  Files  and  Make 

Air  Terminal  Inventories 

60531;  Ilf,  g 

1.0  , 

Mobile  Boarding  Staircase 

60530;  lla(4),  llb(4) 

1.0 
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Table  3-4  (Continued) 


Related 

Estimated 

Lesson  Topics 

STS  Items 

Hours 

Palletization  of  Cargo  on  463L 

Pallets 

60531;  10j(l)-(4) 

1.0 

Operate  Scales  to  Weight  Palletized 
Cargo 

60531;  10j(5),  (6) 

1.0 

Mobile  Baggage  Conveyor 

60530;  lla(5),  llb(5) 

1.0 

Assemble  Cargo  and  Mall  for  Loading 
Using  Pre-Load  Documentation 

60531;  12a 

1.0 

Manifesting 

60531;  llj(l)-(4) 

3.0 

Procedures  for  Loading  and 

Offloading 

60531;  12d(l)-(7) 

1.0 

Install/Remove  Tie-Oowns;  Secure 

Cargo  and  Mall 

60531;  12e(2),  (3) 

1.0 

Briefing  Aircrews 

60531;  11c 

1.0 

4000-Pound  Forklift 

60531;  8a(2),  8b(2) 

1.0 

Warehouse  Tug 

60530;  lla(l),  llb(l), 
60531;  8a(l),  8b(l) 

1.0 

Types  and  Methods  of  Airdrop 

60531;  14b 

1.0 

Types  of  Airdrop  Containers 

60531;  14c 

1.0 

Types  of  Load  Platforms 

60531;  14d 

1.0 

Types  of  Extraction  Systems 

60531;  14e 

1.0 

Types  of  Parachutes 

60531;  14a 

1.0 

Airlift 

60531;  6a,  b 

1.0 

-18  Air  Conditioner  (65  Ton) 

60530;  lla(6),  llb(6) 

1.0 

Records,  Reports,  and  Forms 

60531;  9a-f 

5.0 

Records,  Reports,  and  Forms 

60530;  10 

1.0 

TOTAL  51.0 

Table  3-5 


Potential  Topics  for  Phase  II 

CBT  Development  by  ISO,  Inc. 

Lesson  Topics 

Related 

STS  Items 

Estimated 

Hours 

Organization  and  Mission  of  MAC 

60531;  5 

2.0 

MAC  Mobility  Concept 

60531;  7a 

1.0 

lOK  Forklift  Operator  Inspec¬ 
tion,  Maintenance,  and  Safety 
Procedures 

60531;  8a(3),  8b(3), 

60530;  lla(2),  llb(2) 

5.0 

Adverse  Terrain  Forklift  (ATL) 

60531;  8a(4),  8b(4) 

6.0 

lOK  AT  Forklift  (TEREX) 

Operator  Maintenance  and 

Safety 

lOK  AT  Forklift  (CASE) 

Operator  Maintenance  and 

Safety 

25K  Loader  Operator  Maintenance, 
Inspection  and  Safety  Procedures 

60531;  8a(5),  8b(5) 

5.0 

40K  Loader  Operator  Maintenance, 
Inspection  and  Safety  Procedures 

60531;  8a(6),  8b(6) 

5.0 

H-19  TAC  Loader 

60531;  8a (7),  8b(7) 

6.0 

Inflight  Meal  Procedures 

60530;  9f 

3.0 

Perform  Cargo/Mail  Acceptance 
and  Inspection 

60531;  10a(2),  10a(5) 

4.0 

Inspect  Shipments  for  Pilferage, 
Packaging,  and  Documentation 

60531:  10a(6),  10a(7), 
10a(8) 

2.0 

Hazardous  Cargo  Handler 

60531;  10a(3),  10a(4) 

9.0 

Transportation  Control  and  Move¬ 
ment  Document  (TCMD) 

60531;  lOd 

8.0 

Traffic  Transfer  Receipt 

AF  Form  127 

60531;  lOe 

4.0 
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Table  3-5  (Continued) 


Lesson  Topics 

Related 

STS  Items 

Estimated 

Hours 

In-Processlng/Termlnatlng  Cargo 

60531;  101 

7.0 

Master  463L  Pallet 

60531:  10m 

3.0 

Load  Message  AM-9 

60531;  11b 

3.0 

Load  Planning 

60531;  llh 

9.0 

Aircraft  Load  Shoring 

60531;  12b 

5.0 

Cargo  Tie-Down  and  Restraint 

60531;  12e(l),  12e(2) 

5.0 

TOTAL  92.0 

